Optimal scaling of the Random Walk Metropolis algorithm under Lp mean differentiability

TL;DR

This paper investigates the optimal scaling of high-dimensional random walk Metropolis algorithms for densities that are Lp mean differentiable but may be irregular, establishing weak convergence to a potentially singular Langevin diffusion.

Contribution

It extends the optimal scaling theory to densities with weaker differentiability conditions, including irregular and support-restricted cases, with implications for Bayesian methods.

Findings

Weak convergence of the rescaled Markov chain to a Langevin diffusion.

Applicable to densities with non-differentiable points and support constraints.

Provides practical guidelines for high-dimensional Bayesian computation.

Abstract

This paper considers the optimal scaling problem for high-dimensional random walk Metropolis algorithms for densities which are differentiable in Lp mean but which may be irregular at some points (like the Laplace density for example) and/or are supported on an interval. Our main result is the weak convergence of the Markov chain (appropriately rescaled in time and space) to a Langevin diffusion process as the dimension d goes to infinity. Because the log-density might be non-differentiable, the limiting diffusion could be singular. The scaling limit is established under assumptions which are much weaker than the one used in the original derivation of [6]. This result has important practical implications for the use of random walk Metropolis algorithms in Bayesian frameworks based on sparsity inducing priors.