Latent ODEs for Irregularly-Sampled Time Series

TL;DR

This paper introduces Latent ODEs and ODE-RNNs, models that handle irregularly-sampled time series by defining continuous-time dynamics, outperforming traditional RNNs on such data.

Contribution

The paper generalizes RNNs to continuous-time models using ODEs and integrates them into Latent ODEs, enabling better modeling of irregular time series.

Findings

ODE-based models outperform RNNs on irregular data

Models explicitly handle arbitrary time gaps

Poisson processes model observation times effectively

Abstract

Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace the recognition network of the recently-proposed Latent ODE model. Both ODE-RNNs and Latent ODEs can naturally handle arbitrary time gaps between observations, and can explicitly model the probability of observation times using Poisson processes. We show experimentally that these ODE-based models outperform their RNN-based counterparts on irregularly-sampled data.