Autocovariance and Optimal Design for Random Walk Metropolis-Hastings Algorithm

TL;DR

This paper investigates the covariance structure of the Random Walk Metropolis-Hastings algorithm, providing new theoretical insights and optimal proposal design, especially for symmetric unimodal targets and high-dimensional settings.

Contribution

It offers the first detailed analysis of the covariance structure and proposes an optimal proposal design for the scalar case, extending results to high dimensions.

Findings

Established a theoretical link between covariance and acceptance rate.

Derived optimal proposal parameters for symmetric unimodal targets.

Connected covariance properties to the 0.23 acceptance rate heuristic.

Abstract

The Metropolis-Hastings algorithm has been extensively studied in the estimation and simulation literature, with most prior work focusing on convergence behavior and asymptotic theory. However, its covariance structure-an important statistical property for both theory and implementation-remains less understood. In this work, we provide new theoretical insights into the scalar case, focusing primarily on symmetric unimodal target distributions with symmetric random walk proposals, where we also establish an optimal proposal design. In addition, we derive some more general results beyond this setting. For the high-dimensional case, we relate the covariance matrix to the classical 0.23 average acceptance rate tuning criterion.