Positive autocorrelation at unit lag for stationary random walk Metropolis-Hastings in ${\mathbb R}^d$

TL;DR

This paper proves that stationary random walk Metropolis-Hastings chains in Euclidean space exhibit positive autocorrelation at lag one for any nonzero linear functional, with explicit bounds under symmetric unimodal target distributions.

Contribution

It establishes the strict positivity of autocorrelation at lag one for stationary RWMH in ${f R}^d$, providing bounds and analyzing autocorrelation properties under specific distributional assumptions.

Findings

Autocorrelation at lag one is strictly positive for any nonzero vector.

Lower bound of 1/9 on autocorrelation for spherically symmetric unimodal targets.

Results extend to integer grid state spaces with weak inequality.

Abstract

It is often asserted in the literature that one should expect positive autocorrelation for random walk Metropolis-Hastings (RWMH), especially if the typical proposal step-size is small relative to the variability in the target density. In this paper, we consider a stationary RWMH chain ${\bf X}$ taking values in $d$-dimensional Euclidean space and (subject only to the existence of densities with respect to Lebesgue measure) with general target distribution having finite second moment and general proposal random walk step-distribution. We prove, for any nonzero vector ${\bf c}$, strict positivity of the autocorrelation function at unit lag for the stochastic process $\langle{\bf c},{\bf X}\rangle$, that is, \[{\operatorname{Corr}}(\langle{\bf c},{\bf X}_0\rangle,\langle{\bf c},{\bf X}_1\rangle)>0,\] and we establish the same result, but with weak inequality (which can in some cases be equality) when the state space for ${\bf X}$ is changed to the integer grid ${\mathbb Z}^d$. Further, for ${\bf c}\neq{\bf 0}$ we establish the sharp lower bound \[{\operatorname{Corr}}(\langle{\bf c},{\bf X}_0\rangle,\langle{\bf c},{\bf X}_1\rangle)>\tfrac19\] on autocorrelation when we assume both that (i) the target density $\pi$ is spherically symmetric and unimodal in the specific sense that $\pi({\bf x})=\hat{\pi}(\|{\bf x}\|)$ for some nonincreasing function $\hat{\pi}$ on $[0,\infty)$ and that (ii) the proposal step-density is symmetric about ${\bf 0}$. We study the autocorrelation indirectly, by considering the incremental variance function (or incremental second-moment function) at unit lag. The same approach allows us also for $r\in[2,\infty)$ to upper-bound the incremental $r$th-absolute-moment function at unit lag. We give also closely related inequalities for the total variation distance between two distributions on ${\mathbb R}^d$ differing only by a location shift.