GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series

TL;DR

GRU-ODE-Bayes introduces a continuous-time recurrent model with Bayesian updates for irregularly observed time series, excelling in healthcare and climate data analysis by capturing complex dynamics and performing well with limited samples.

Contribution

It combines Neural ODEs with Bayesian updates to model sporadically observed time series, providing a novel continuous-time recurrent framework with theoretical and empirical advantages.

Findings

Outperforms state-of-the-art methods on synthetic and real-world data.

Encodes a continuity prior suitable for low-sample scenarios.

Can exactly represent Fokker-Planck dynamics of complex stochastic processes.

Abstract

Modeling real-world multidimensional time series can be particularly challenging when these are sporadically observed (i.e., sampling is irregular both in time and across dimensions)-such as in the case of clinical patient data. To address these challenges, we propose (1) a continuous-time version of the Gated Recurrent Unit, building upon the recent Neural Ordinary Differential Equations (Chen et al., 2018), and (2) a Bayesian update network that processes the sporadic observations. We bring these two ideas together in our GRU-ODE-Bayes method. We then demonstrate that the proposed method encodes a continuity prior for the latent process and that it can exactly represent the Fokker-Planck dynamics of complex processes driven by a multidimensional stochastic differential equation. Additionally, empirical evaluation shows that our method outperforms the state of the art on both synthetic data and real-world data with applications in healthcare and climate forecast. What is more, the continuity prior is shown to be well suited for low number of samples settings.