Bayesian Gaussian Process ODEs via Double Normalizing Flows

TL;DR

This paper introduces a novel approach combining normalizing flows with Gaussian process ODEs to enhance model flexibility, accuracy, and uncertainty estimation in dynamical systems modeling, validated on simulated and real-world data.

Contribution

We propose integrating normalizing flows into Bayesian GP ODEs to increase expressiveness and improve inference, addressing limitations of standard kernels and mean-field assumptions.

Findings

Enhanced accuracy in time series prediction

Improved uncertainty quantification

Effective modeling of complex dynamical systems

Abstract

Recently, Gaussian processes have been used to model the vector field of continuous dynamical systems, referred to as GPODEs, which are characterized by a probabilistic ODE equation. Bayesian inference for these models has been extensively studied and applied in tasks such as time series prediction. However, the use of standard GPs with basic kernels like squared exponential kernels has been common in GPODE research, limiting the model's ability to represent complex scenarios. To address this limitation, we introduce normalizing flows to reparameterize the ODE vector field, resulting in a data-driven prior distribution, thereby increasing flexibility and expressive power. We develop a data-driven variational learning algorithm that utilizes analytically tractable probability density functions of normalizing flows, enabling simultaneous learning and inference of unknown continuous dynamics. Additionally, we also apply normalizing flows to the posterior inference of GP ODEs to resolve the issue of strong mean-field assumptions in posterior inference. By applying normalizing flows in both these ways, our model improves accuracy and uncertainty estimates for Bayesian Gaussian Process ODEs. We validate the effectiveness of our approach on simulated dynamical systems and real-world human motion data, including time series prediction and missing data recovery tasks. Experimental results show that our proposed method effectively captures model uncertainty while improving accuracy.