A note on diffusive/random-walk behaviour in Metropolis--Hastings algorithms

TL;DR

This paper investigates the conditions under which Metropolis--Hastings algorithms exhibit diffusive or random-walk behavior, establishing links between proposal properties, acceptance rates, and ergodicity, with implications for convergence speed.

Contribution

It provides necessary conditions for non-geometric ergodicity based on proposal and acceptance rate behaviors, and compares convergence rates of different Metropolis algorithms under various tail conditions.

Findings

Non-geometric ergodicity persists if proposal is not geometrically ergodic and acceptance rate approaches unity.

Guided walk converges faster than random walk for polynomial tail distributions.

Random walk behaves as a lazy version of guided walk under convex potential, moving at similar speed.

Abstract

We prove a general result that if a Metropolis--Hastings algorithm has a proposal that is not geometrically ergodic and the acceptance rate approaches unity at a suitable rate as the state variable becomes large, then the Metropolised chain will also not be geometrically ergodic. Our conditions seem stronger than might be expected, but are shown to be necessary through a counterexample. We then turn our attention to the random walk and guided walk Metropolis algorithms. We show that if the target distribution has polynomial tails the latter converges at twice the polynomial rate of the former, but that if instead the target distribution has strictly convex potential then the random walk Metropolis behaves as a $1/2$-lazy version of the guided walk Metropolis when the state variable is large, and therefore moves at a similar (ballistic) speed.