Bayesian Nonparametric Mixed-Effect ODEs with Gaussian Processes

TL;DR

This paper introduces MEGPODE, a Bayesian nonparametric mixed-effect ODE model using Gaussian processes to better capture heterogeneity in dynamical systems across subjects.

Contribution

It develops a novel hierarchical Gaussian process framework for mixed-effect ODEs that avoids repeated solves and improves prediction accuracy.

Findings

MEGPODE outperforms strong baselines in heterogeneous ODE benchmarks.

The model effectively recovers population and subject-specific dynamics.

Closed-form regressions enable efficient training without repeated ODE solves.

Abstract

Dynamical modelling is central to many scientific domains, including pharmacometrics, systems biology, physiology, and epidemiology. In these settings, heterogeneity is often intrinsic: different subjects or units follow related but distinct continuous-time dynamics. Classical nonlinear mixed-effects Ordinary Differential Equation (ODE) models address this by combining population-level structure with subject-specific effects, but they rely on a parametric vector field and are therefore vulnerable to structural misspecification and unmodelled mechanisms. This motivates nonparametric approaches that can retain principled uncertainty quantification, yet existing nonparametric ODE methods typically assume a single shared dynamical system rather than an explicit mixed-effect hierarchy over subject-specific dynamics. We propose MEGPODE, a Bayesian nonparametric mixed-effect ODE model in which each subject's vector field is decomposed into a shared population component and a subject-specific deviation, both endowed with Gaussian process (GP) priors. To avoid repeated ODE solves per subject during training, we combine state-space GP trajectory priors with virtual collocation observations, yielding Kalman-smoothing trajectory updates and closed-form regressions for the vector fields. Across controlled heterogeneous ODE benchmarks spanning oscillatory, biomedical systems, MEGPODE improves population-field recovery and subject-level trajectory prediction relative to strong baselines.