Geometric ergodicity of the Random Walk Metropolis with position-dependent proposal covariance

TL;DR

This paper investigates how using a position-dependent covariance matrix in the Metropolis-Hastings algorithm affects its ergodicity, showing that tailored choices can improve convergence properties especially in complex tail behaviors.

Contribution

It demonstrates that adaptive covariance matrices can enhance geometric ergodicity of the Metropolis-Hastings algorithm under various tail conditions, extending previous results for fixed proposals.

Findings

Unbounded growth of proposal variance in tails can ensure geometric ergodicity for heavy-tailed distributions.

Properly chosen $G(x)$ can achieve geometric ergodicity on narrow ridges, unlike standard RWM.

Growth rate of proposal variance must be controlled to avoid high rejection rates.

Abstract

We consider a Metropolis--Hastings method with proposal $\mathcal{N}(x, hG(x)^{-1})$, where $x$ is the current state, and study its ergodicity properties. We show that suitable choices of $G(x)$ can change these compared to the Random Walk Metropolis case $\mathcal{N}(x, h\Sigma)$, either for better or worse. We find that if the proposal variance is allowed to grow unboundedly in the tails of the distribution then geometric ergodicity can be established when the target distribution for the algorithm has tails that are heavier than exponential, but that the growth rate must be carefully controlled to prevent the rejection rate approaching unity. We also illustrate that a judicious choice of $G(x)$ can result in a geometrically ergodic chain when probability concentrates on an ever narrower ridge in the tails, something that is not true for the Random Walk Metropolis.