Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series

TL;DR

This paper introduces NCDSSM, a novel continuous-time state space model that effectively handles irregularly-sampled time series with missing data, improving forecasting and imputation accuracy across diverse domains.

Contribution

The paper presents NCDSSM, a new neural state space model that combines continuous-discrete filtering with auxiliary variables for efficient inference on irregular time series.

Findings

Outperforms existing models in imputation tasks

Achieves superior forecasting accuracy on benchmark datasets

Demonstrates flexibility with three latent dynamics parameterizations

Abstract

Learning accurate predictive models of real-world dynamic phenomena (e.g., climate, biological) remains a challenging task. One key issue is that the data generated by both natural and artificial processes often comprise time series that are irregularly sampled and/or contain missing observations. In this work, we propose the Neural Continuous-Discrete State Space Model (NCDSSM) for continuous-time modeling of time series through discrete-time observations. NCDSSM employs auxiliary variables to disentangle recognition from dynamics, thus requiring amortized inference only for the auxiliary variables. Leveraging techniques from continuous-discrete filtering theory, we demonstrate how to perform accurate Bayesian inference for the dynamic states. We propose three flexible parameterizations of the latent dynamics and an efficient training objective that marginalizes the dynamic states during inference. Empirical results on multiple benchmark datasets across various domains show improved imputation and forecasting performance of NCDSSM over existing models.