Neural ODE Processes

TL;DR

Neural ODE Processes (NDPs) combine Neural ODEs and Neural Processes to enable uncertainty estimation and real-time adaptation in modeling complex, high-dimensional time-series dynamics from limited data.

Contribution

The paper introduces Neural ODE Processes, a novel stochastic process framework that captures dynamics, uncertainty, and adapts quickly to new data, overcoming limitations of existing models.

Findings

NDPs effectively model low-dimensional dynamics with few data points.

NDPs scale to high-dimensional time-series like rotating MNIST.

NDPs provide uncertainty estimates and real-time data adaptation.

Abstract

Neural Ordinary Differential Equations (NODEs) use a neural network to model the instantaneous rate of change in the state of a system. However, despite their apparent suitability for dynamics-governed time-series, NODEs present a few disadvantages. First, they are unable to adapt to incoming data points, a fundamental requirement for real-time applications imposed by the natural direction of time. Second, time series are often composed of a sparse set of measurements that could be explained by many possible underlying dynamics. NODEs do not capture this uncertainty. In contrast, Neural Processes (NPs) are a family of models providing uncertainty estimation and fast data adaptation but lack an explicit treatment of the flow of time. To address these problems, we introduce Neural ODE Processes (NDPs), a new class of stochastic processes determined by a distribution over Neural ODEs. By maintaining an adaptive data-dependent distribution over the underlying ODE, we show that our model can successfully capture the dynamics of low-dimensional systems from just a few data points. At the same time, we demonstrate that NDPs scale up to challenging high-dimensional time-series with unknown latent dynamics such as rotating MNIST digits.