Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Series

TL;DR

This paper introduces ACSSM, a novel method for modeling irregular time series using continuous state space models, stochastic optimal control, and amortized inference, achieving superior performance and efficiency on real-world datasets.

Contribution

The paper presents ACSSM, combining Doob's $h$-transform, variational inference, and amortized control for scalable, accurate modeling of irregular time series.

Findings

Outperforms existing methods in classification, regression, interpolation, and extrapolation.

Demonstrates computational efficiency and scalability on real-world datasets.

Effectively models irregular and discrete observations in continuous dynamical systems.

Abstract

Many real-world datasets, such as healthcare, climate, and economics, are often collected as irregular time series, which poses challenges for accurate modeling. In this paper, we propose the Amortized Control of continuous State Space Model (ACSSM) for continuous dynamical modeling of time series for irregular and discrete observations. We first present a multi-marginal Doob's $h$-transform to construct a continuous dynamical system conditioned on these irregular observations. Following this, we introduce a variational inference algorithm with a tight evidence lower bound (ELBO), leveraging stochastic optimal control (SOC) theory to approximate the intractable Doob's $h$-transform and simulate the conditioned dynamics. To improve efficiency and scalability during both training and inference, ACSSM leverages auxiliary variable to flexibly parameterize the latent dynamics and amortized control. Additionally, it incorporates a simulation-free latent dynamics framework and a transformer-based data assimilation scheme, facilitating parallel inference of the latent states and ELBO computation. Through empirical evaluations across a variety of real-world datasets, ACSSM demonstrates superior performance in tasks such as classification, regression, interpolation, and extrapolation, while maintaining computational efficiency.