Maximum Likelihood Learning of Latent Dynamics Without Reconstruction

TL;DR

This paper presents RP-GSSM, an unsupervised probabilistic model for time series with latent dynamics, enabling maximum likelihood learning without explicit reconstruction, and demonstrating superior performance on nonlinear stochastic dynamics tasks.

Contribution

The paper introduces RP-GSSM, a novel probabilistic model that combines tractability with expressivity, allowing maximum likelihood learning of latent dynamics without reconstruction.

Findings

Outperforms existing methods on nonlinear stochastic dynamics from video

Enables exact inference with Gaussian latent priors

Learns task-relevant latents without ad-hoc regularization

Abstract

We introduce a novel unsupervised learning method for time series data with latent dynamical structure: the recognition-parametrized Gaussian state space model (RP-GSSM). The RP-GSSM is a probabilistic model that learns Markovian Gaussian latents explaining statistical dependence between observations at different time steps, combining the intuition of contrastive methods with the flexible tools of probabilistic generative models. Unlike contrastive approaches, the RP-GSSM is a valid probabilistic model learned via maximum likelihood. Unlike generative approaches, the RP-GSSM has no need for an explicit network mapping from latents to observations, allowing it to focus model capacity on inference of latents. The model is both tractable and expressive: it admits exact inference thanks to its jointly Gaussian latent prior, while maintaining expressivity with an arbitrarily nonlinear neural network link between observations and latents. These qualities allow the RP-GSSM to learn task-relevant latents without ad-hoc regularization, auxiliary losses, or optimizer scheduling. We show how this approach outperforms alternatives on problems that include learning nonlinear stochastic dynamics from video, with or without background distractors. Our results position the RP-GSSM as a useful foundation model for a variety of downstream applications.