Self-Supervised Inference in State-Space Models

TL;DR

This paper introduces a neural network-based method for approximate inference in nonlinear state-space models that does not require supervised data or explicit generative models, leveraging local linearity and Bayesian updates.

Contribution

It presents a novel approach combining neural networks with Bayesian updates for inference in nonlinear state-space models without supervision or explicit generative modeling.

Findings

Achieved excellent results on the Lorenz system using domain knowledge.

Demonstrated competitive performance on audio denoising.

Method simplifies inference while maintaining accuracy.

Abstract

We perform approximate inference in state-space models with nonlinear state transitions. Without parameterizing a generative model, we apply Bayesian update formulas using a local linearity approximation parameterized by neural networks. This comes accompanied by a maximum likelihood objective that requires no supervision via uncorrupt observations or ground truth latent states. The optimization backpropagates through a recursion similar to the classical Kalman filter and smoother. Additionally, using an approximate conditional independence, we can perform smoothing without having to parameterize a separate model. In scientific applications, domain knowledge can give a linear approximation of the latent transition maps, which we can easily incorporate into our model. Usage of such domain knowledge is reflected in excellent results (despite our model's simplicity) on the chaotic Lorenz system compared to fully supervised and variational inference methods. Finally, we show competitive results on an audio denoising experiment.