Optimal scaling of random-walk Metropolis algorithms using Bayesian large-sample asymptotics

TL;DR

This paper investigates optimal scaling for high-dimensional random-walk Metropolis algorithms using Bayesian large-sample asymptotics, providing practical tuning guidelines that account for correlation structures in target densities.

Contribution

It introduces a large-sample perspective to derive realistic, dimension-dependent tuning rules for Metropolis algorithms, extending beyond restrictive product-form assumptions.

Findings

Proposes new parameter-dimension-dependent tuning guidelines.

Shows correlation structure impacts algorithm performance.

Validates guidelines through theoretical and practical analysis.

Abstract

High-dimensional limit theorems have been shown useful to derive tuning rules for finding the optimal scaling in random-walk Metropolis algorithms. The assumptions under which weak convergence results are proved are however restrictive: the target density is typically assumed to be of a product form. Users may thus doubt the validity of such tuning rules in practical applications. In this paper, we shed some light on optimal-scaling problems from a different perspective, namely a large-sample one. This allows to prove weak convergence results under realistic assumptions and to propose novel parameter-dimension-dependent tuning guidelines. The proposed guidelines are consistent with previous ones when the target density is close to having a product form, and the results highlight that the correlation structure has to be accounted for to avoid performance deterioration if that is not the case, while justifying the use of a natural (asymptotically exact) approximation to the correlation matrix that can be employed for the very first algorithm run.