Exploring the generalizability of the optimal 0.234 acceptance rate in random-walk Metropolis and parallel tempering algorithms

TL;DR

This paper investigates the robustness and practical applicability of the optimal 0.234 acceptance rate in random-walk Metropolis and parallel tempering algorithms through theoretical synthesis and extensive empirical simulations.

Contribution

It extends the understanding of the 0.234 acceptance rate's relevance beyond high-dimensional limits and demonstrates its effectiveness in various practical, lower-dimensional, and complex target distribution settings.

Findings

0.234 acceptance rate is robust in lower dimensions

Optimal inverse temperature spacing with 0.234 swap acceptance is effective

Theoretical extensions support practical use of 0.234 in diverse scenarios

Abstract

For random-walk Metropolis (RWM) and parallel tempering (PT) algorithms, an asymptotic acceptance rate of around 0.234 is known to be optimal in certain high-dimensional limits. However, its practical relevance is uncertain due to restrictive derivation conditions. We synthesise previous theoretical advances in extending the 0.234 acceptance rate to more general settings, and demonstrate its applicability with a comprehensive empirical simulation study on examples examining how acceptance rates affect Expected Squared Jumping Distance (ESJD). Our experiments show the optimality of the 0.234 acceptance rate for RWM is surprisingly robust even in lower dimensions across various non-spherically symmetric proposal distributions, multimodal target distributions that may not have an i.i.d. product density, and curved Rosenbrock target distributions with nonlinear correlation structure. Parallel tempering experiments also show that the idealized 0.234 spacing of inverse temperatures may be approximately optimal for low dimensions and non i.i.d. product target densities, and that constructing an inverse temperature ladder with spacings given by a swap acceptance of 0.234 is a viable strategy.