SING: SDE Inference via Natural Gradients

TL;DR

SING introduces a natural gradient variational inference method for latent SDEs, enabling faster, more stable, and more accurate inference of dynamical systems, with applications in neuroscience and beyond.

Contribution

The paper presents SING, a novel natural gradient-based inference method for latent SDEs that improves convergence speed, stability, and accuracy over existing approaches.

Findings

SING achieves faster convergence than prior methods.

SING provides more accurate state and drift function estimates.

Application to neural data demonstrates practical effectiveness.

Abstract

Latent stochastic differential equation (SDE) models are important tools for the unsupervised discovery of dynamical systems from data, with applications ranging from engineering to neuroscience. In these complex domains, exact posterior inference of the latent state path is typically intractable, motivating the use of approximate methods such as variational inference (VI). However, existing VI methods for inference in latent SDEs often suffer from slow convergence and numerical instability. We propose SDE Inference via Natural Gradients (SING), a method that leverages natural gradient VI to efficiently exploit the underlying geometry of the model and variational posterior. SING enables fast and reliable inference in latent SDE models by approximating intractable integrals and parallelizing computations in time. We provide theoretical guarantees that SING approximately optimizes the intractable, continuous-time objective of interest. Moreover, we demonstrate that better state inference enables more accurate estimation of nonlinear drift functions using, for example, Gaussian process SDE models. SING outperforms prior methods in state inference and drift estimation on a variety of datasets, including a challenging application to modeling neural dynamics in freely behaving animals. Altogether, our results illustrate the potential of SING as a tool for accurate inference in complex dynamical systems, especially those characterized by limited prior knowledge and non-conjugate structure.