Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems

TL;DR

This paper introduces auxiliary MCMC and particle Gibbs samplers that improve parallelisable inference in high-dimensional latent dynamical systems, addressing performance degradation of existing methods.

Contribution

The authors develop novel auxiliary sampling methods incorporating artificial observations, enabling efficient, parallelisable inference in high-dimensional models with improved performance.

Findings

Superior statistical performance over existing methods

Enhanced computational efficiency, especially with GPU parallelisation

Maintains performance in high-dimensional latent spaces

Abstract

Sampling from the full posterior distribution of high-dimensional non-linear, non-Gaussian latent dynamical models presents significant computational challenges. While Particle Gibbs (also known as conditional sequential Monte Carlo) is considered the gold standard for this task, it quickly degrades in performance as the latent space dimensionality increases. Conversely, globally Gaussian-approximated methods like extended Kalman filtering, though more robust, are seldom used for posterior sampling due to their inherent bias. We introduce novel auxiliary sampling approaches that address these limitations. By incorporating artificial observations of the system as auxiliary variables in our MCMC kernels, we develop both efficient exact Kalman-based samplers and enhanced Particle Gibbs algorithms that maintain performance in high-dimensional latent spaces. Some of our methods support parallelisation along the time dimension, achieving logarithmic scaling when implemented on GPUs. Empirical evaluations demonstrate superior statistical and computational performance compared to existing approaches for high-dimensional latent dynamical systems.