Path-minimizing Latent ODEs for improved extrapolation and inference

TL;DR

This paper introduces a path-length regularization technique for latent ODE models, enhancing their ability to extrapolate and infer complex dynamics more accurately and efficiently.

Contribution

The authors propose replacing the variational penalty with an $ ext{L}_2$ path length penalty, leading to faster training, smaller models, and improved extrapolation in latent ODEs.

Findings

Faster training and smaller models compared to baseline ODE models.

More accurate long-term extrapolation of dynamic systems.

Superior performance in simulation-based inference tasks.

Abstract

Latent ODE models provide flexible descriptions of dynamic systems, but they can struggle with extrapolation and predicting complicated non-linear dynamics. The latent ODE approach implicitly relies on encoders to identify unknown system parameters and initial conditions, whereas the evaluation times are known and directly provided to the ODE solver. This dichotomy can be exploited by encouraging time-independent latent representations. By replacing the common variational penalty in latent space with an $\ell_2$ penalty on the path length of each system, the models learn data representations that can easily be distinguished from those of systems with different configurations. This results in faster training, smaller models, more accurate interpolation and long-time extrapolation compared to the baseline ODE models with GRU, RNN, and LSTM encoder/decoders on tests with damped harmonic oscillator, self-gravitating fluid, and predator-prey systems. We also demonstrate superior results for simulation-based inference of the Lotka-Volterra parameters and initial conditions by using the latents as data summaries for a conditional normalizing flow. Our change to the training loss is agnostic to the specific recognition network used by the decoder and can therefore easily be adopted by other latent ODE models.