Variational Neural Stochastic Differential Equations with Change Points

TL;DR

This paper introduces a variational neural SDE framework with change point detection capabilities, enabling effective modeling of time-series data with distribution shifts using a novel training approach.

Contribution

It proposes a new neural SDE model with a simplified prior and develops two methods for detecting change points, enhancing modeling of non-stationary time-series.

Findings

Effective modeling of classical parametric SDEs

Successful detection of distribution shifts in real datasets

Theoretical validation of change point detection method

Abstract

In this work, we explore modeling change points in time-series data using neural stochastic differential equations (neural SDEs). We propose a novel model formulation and training procedure based on the variational autoencoder (VAE) framework for modeling time-series as a neural SDE. Unlike existing algorithms training neural SDEs as VAEs, our proposed algorithm only necessitates a Gaussian prior of the initial state of the latent stochastic process, rather than a Wiener process prior on the entire latent stochastic process. We develop two methodologies for modeling and estimating change points in time-series data with distribution shifts. Our iterative algorithm alternates between updating neural SDE parameters and updating the change points based on either a maximum likelihood-based approach or a change point detection algorithm using the sequential likelihood ratio test. We provide a theoretical analysis of this proposed change point detection scheme. Finally, we present an empirical evaluation that demonstrates the expressive power of our proposed model, showing that it can effectively model both classical parametric SDEs and some real datasets with distribution shifts.