Asymptotic variance for Random walk Metropolis chains in high dimensions: logarithmic growth via the Poisson equation

TL;DR

This paper establishes a theoretical link between the growth of asymptotic variance in high-dimensional Random walk Metropolis algorithms and the dimension, showing it grows logarithmically with dimension using the Poisson equation.

Contribution

It provides the first rigorous analysis connecting the asymptotic variance growth to dimension for control variates in high-dimensional RWM chains, using the Poisson equation and advanced probabilistic tools.

Findings

Asymptotic variance is bounded by a multiple of log(d)/d over the spectral gap.

The proof employs large deviations, Young's inequality, and Berry-Esseen bounds.

Extensions to non-product targets are discussed.

Abstract

There are two ways of speeding up MCMC algorithms: (1) construct more complex samplers that use gradient and higher order information about the target and (2) design a control variate to reduce the asymptotic variance. While the efficiency of (1) as a function of dimension has been studied extensively, this paper provides first rigorous results linking the growth of the asymptotic variance in (2) with dimension. Specifically, we construct a control variate for a $d$-dimensional Random walk Metropolis chain with an IID target using the solution of the Poisson equation for the scaling limit in the seminal paper "Weak convergence and optimal scaling of random walk Metropolis algorithms" of Gelman, Gilks and Roberts. We prove that the asymptotic variance of the corresponding estimator is bounded above by a multiple of $\log d/d$ over the spectral gap of the chain. The proof hinges on large deviations theory, optimal Young's inequality and Berry-Esseen type bounds. Extensions of the result to non-product targets are discussed.