System-Aware Neural ODE Processes for Few-Shot Bayesian Optimization

TL;DR

This paper introduces SANODEP, a system-aware neural ODE process that enhances few-shot Bayesian optimization for dynamical systems with delayed measurements and limited trials.

Contribution

The paper proposes SANODEP, a novel extension of Neural ODE Processes that meta-learns ODE systems using a new context embedding, improving few-shot Bayesian optimization in complex dynamical systems.

Findings

SANODEP effectively models ODE systems from limited trajectories.

The two-stage BO framework improves optimization efficiency with constraints.

SANODEP adapts to different levels of prior information.

Abstract

We consider the problem of optimizing initial conditions and termination time in dynamical systems governed by unknown ordinary differential equations (ODEs), where evaluating different initial conditions is costly and the state's value can not be measured in real-time but only with a delay while the measuring device processes the sample. To identify the optimal conditions in limited trials, we introduce a few-shot Bayesian Optimization (BO) framework based on the system's prior information. At the core of our approach is the System-Aware Neural ODE Processes (SANODEP), an extension of Neural ODE Processes (NODEP) designed to meta-learn ODE systems from multiple trajectories using a novel context embedding block. We further develop a two-stage BO framework to effectively incorporate search space constraints, enabling efficient optimization of both initial conditions and observation timings. We conduct extensive experiments showcasing SANODEP's potential for few-shot BO within dynamical systems. We also explore SANODEP's adaptability to varying levels of prior information, highlighting the trade-off between prior flexibility and model fitting accuracy.