Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations

TL;DR

This paper introduces a fully Bayesian approach to differential Gaussian processes using stochastic differential equations, modeling hyperparameters as random variables to improve flexibility and predictive performance in dynamic systems.

Contribution

It presents a novel Bayesian framework that jointly learns hyperparameters and inducing points using coupled SDEs, enhancing model expressiveness and uncertainty quantification.

Findings

Outperforms traditional methods in accuracy and flexibility

Effectively models complex dynamic behaviors

Improves posterior approximation with neural network-based SDE solver

Abstract

Deep Gaussian process models typically employ discrete hierarchies, but recent advancements in differential Gaussian processes (DiffGPs) have extended these models to infinite depths. However, existing DiffGP approaches often overlook the uncertainty in kernel hyperparameters by treating them as fixed and time-invariant, which degrades the model's predictive performance and neglects the posterior distribution. In this work, we introduce a fully Bayesian framework that models kernel hyperparameters as random variables and utilizes coupled stochastic differential equations (SDEs) to jointly learn their posterior distributions alongside those of inducing points. By incorporating the estimation uncertainty of hyperparameters, our method significantly enhances model flexibility and adaptability to complex dynamic systems. Furthermore, we employ a black-box adaptive SDE solver with a neural network to achieve realistic, time varying posterior approximations, thereby improving the expressiveness of the variational posterior. Comprehensive experimental evaluations demonstrate that our approach outperforms traditional methods in terms of flexibility, accuracy, and other key performance metrics. This work not only provides a robust Bayesian extension to DiffGP models but also validates its effectiveness in handling intricate dynamic behaviors, thereby advancing the applicability of Gaussian process models in diverse real-world scenarios.