Optimal Scaling and Shaping of Random Walk Metropolis via Diffusion Limits of Block-I.I.D. Targets

TL;DR

This paper analyzes the optimal scaling and shaping of the Random Walk Metropolis algorithm for block-i.i.d. targets, demonstrating the robustness of the 0.234 acceptance rate heuristic and exploring the effects of dependence on performance.

Contribution

It extends existing results to block-i.i.d. targets, confirming the optimal acceptance rate heuristic and identifying the target variance as the optimal shaping measure.

Findings

Optimal acceptance rate is approximately 0.234 for block-i.i.d. targets.

Target variance is the optimal shaping for autocorrelation decay.

Performance degrades with higher-order dependence, limiting RWM effectiveness.

Abstract

This work extends Roberts et al. (1997) by considering limits of Random Walk Metropolis (RWM) applied to block IID target distributions, with corresponding block-independent proposals. The extension verifies the robustness of the optimal scaling heuristic, to tune the acceptance rate to $\approx0.234$, for any choice of proposal shaping. We upgrade the form of weak convergence from a finite-dimensional subprocess to the infinite dimensional process. We show that the optimal shaping (in terms of the decay of autocorrelations of linear functions) is the variance of the target distribution. We show that this choice coincides with the optimal shaping in terms of spectral gaps in special cases where they can be computed. Lastly, we provide some negative guarantees, showing that RWM performance degrades with higher-order dependence. In such cases, no tuning of RWM will yield performance comparable to an IID target.