Gaussian Invariant Markov Chain Monte Carlo

TL;DR

This paper introduces Gaussian invariant versions of popular MCMC algorithms that improve statistical efficiency and variance reduction, especially for high-dimensional Gaussian targets, supported by theoretical analysis and empirical results.

Contribution

It develops Gaussian invariant MCMC methods with analytical solutions for Poisson equations, enabling efficient variance reduction and improved ergodic estimators.

Findings

Gaussian invariant samplers outperform standard methods in high-dimensional settings

Exact solutions to Poisson equations facilitate effective control variates

Theoretical analysis confirms geometric ergodicity and optimal scaling properties.

Abstract

We develop sampling methods, which consist of Gaussian invariant versions of random walk Metropolis (RWM), Metropolis adjusted Langevin algorithm (MALA) and second order Hessian or Manifold MALA. Unlike standard RWM and MALA we show that Gaussian invariant sampling can lead to ergodic estimators with improved statistical efficiency. This is due to a remarkable property of Gaussian invariance that allows us to obtain exact analytical solutions to the Poisson equation for Gaussian targets. These solutions can be used to construct efficient and easy to use control variates for variance reduction of estimators under any intractable target. We demonstrate the new samplers and estimators in several examples, including high dimensional targets in latent Gaussian models where we compare against several advanced methods and obtain state-of-the-art results. We also provide theoretical results regarding geometric ergodicity, and an optimal scaling analysis that shows the dependence of the optimal acceptance rate on the Gaussianity of the target.