Optimal Scaling for the Proximal Langevin Algorithm in High Dimensions

TL;DR

This paper demonstrates that the proximal MALA algorithm maintains optimal scaling and acceptance rates similar to MALA in high dimensions, offering a practical alternative when gradient computation is costly.

Contribution

It proves that proximal MALA achieves the same optimal scaling and acceptance probability as MALA for smooth target densities, extending theoretical understanding.

Findings

Proximal MALA has the same optimal scaling as MALA in high dimensions.

Optimal acceptance probability for proximal MALA is approximately 0.574.

Replacing gradients with proximal functions does not reduce efficiency in high-dimensional sampling.

Abstract

The Metropolis-adjusted Langevin (MALA) algorithm is a sampling algorithm that incorporates the gradient of the logarithm of the target density in its proposal distribution. In an earlier joint work \citet{pill:stu:12}, the author had extended the seminal work of \cite{Robe:Rose:98} and showed that in stationarity, MALA applied to an $N-$dimensional approximation of the target will take ${\cal O}(N^{\frac13})$ steps to explore its target measure. It was also shown that the MALA algorithm is optimized at an average acceptance probability of $0.574$. In \citet{pere:16}, the author introduced the proximal MALA algorithm where the gradient of the log target density is replaced by the proximal function. In this paper, we show that for a wide class of twice differentiable target densities, the proximal MALA enjoys the same optimal scaling as that of MALA in high dimensions and also has an average optimal acceptance probability of $0.574$. The results of this paper thus give the following practically useful guideline: for smooth target densities where it is expensive to compute the gradient while implementing MALA, users may replace the gradient with the corresponding proximal function (that can be often computed relatively cheaply via convex optimization) \emph{without} losing any efficiency gains from optimal scaling. This confirms some of the empirical observations made in \cite{pere:16}.