Linear Variational State-Space Filtering

TL;DR

This paper introduces Variational State-Space Filters (VSSF), a novel unsupervised learning framework for filtering and identifying latent Markov models from raw pixel data, capable of integrating heterogeneous sensor measurements.

Contribution

It presents a theoretically grounded framework for latent state inference with heterogeneous sensors and introduces L-VSSF, a linear, Gaussian instantiation for practical filtering applications.

Findings

L-VSSF can filter in latent space beyond training sequence length.

The method effectively integrates multiple sensor types during training.

Experimental results demonstrate robust filtering across various environments.

Abstract

We introduce Variational State-Space Filters (VSSF), a new method for unsupervised learning, identification, and filtering of latent Markov state space models from raw pixels. We present a theoretically sound framework for latent state space inference under heterogeneous sensor configurations. The resulting model can integrate an arbitrary subset of the sensor measurements used during training, enabling the learning of semi-supervised state representations, thus enforcing that certain components of the learned latent state space to agree with interpretable measurements. From this framework we derive L-VSSF, an explicit instantiation of this model with linear latent dynamics and Gaussian distribution parameterizations. We experimentally demonstrate L-VSSF's ability to filter in latent space beyond the sequence length of the training dataset across several different test environments.