Free-Form Variational Inference for Gaussian Process State-Space Models

TL;DR

This paper introduces a novel free-form variational inference method for Gaussian process state-space models, improving accuracy and computational efficiency in modeling complex dynamic systems.

Contribution

It develops a flexible variational inference approach using stochastic gradient Hamiltonian Monte Carlo, with a collapsed extension and integration with particle MCMC methods.

Findings

More accurate learning of transition dynamics and latent states.

Outperforms competing methods on six real-world datasets.

Reduces computational complexity compared to previous approaches.

Abstract

Gaussian process state-space models (GPSSMs) provide a principled and flexible approach to modeling the dynamics of a latent state, which is observed at discrete-time points via a likelihood model. However, inference in GPSSMs is computationally and statistically challenging due to the large number of latent variables in the model and the strong temporal dependencies between them. In this paper, we propose a new method for inference in Bayesian GPSSMs, which overcomes the drawbacks of previous approaches, namely over-simplified assumptions, and high computational requirements. Our method is based on free-form variational inference via stochastic gradient Hamiltonian Monte Carlo within the inducing-variable formalism. Furthermore, by exploiting our proposed variational distribution, we provide a collapsed extension of our method where the inducing variables are marginalized analytically. We also showcase results when combining our framework with particle MCMC methods. We show that, on six real-world datasets, our approach can learn transition dynamics and latent states more accurately than competing methods.