On the Computational Complexity of Metropolis-Adjusted Langevin Algorithms for Bayesian Posterior Sampling

TL;DR

This paper analyzes the computational complexity of the Metropolis-Adjusted Langevin Algorithm (MALA) for Bayesian posterior sampling, introducing new techniques to handle non-ideal conditions and establishing optimal mixing time bounds.

Contribution

It introduces a novel technique using the $s$-conductance profile to bound mixing times, extending MALA analysis beyond smooth, strongly log-concave targets.

Findings

Established $d^{1/3}$ dimension dependence in mixing time bounds.

Proved a matching lower bound for Gaussian sampling with MALA.

Extended analysis to non-ideal Bayesian posteriors close to Gaussian.

Abstract

In this paper, we examine the computational complexity of sampling from a Bayesian posterior (or pseudo-posterior) using the Metropolis-adjusted Langevin algorithm (MALA). MALA first employs a discrete-time Langevin SDE to propose a new state, and then adjusts the proposed state using Metropolis-Hastings rejection. Most existing theoretical analyses of MALA rely on the smoothness and strong log-concavity properties of the target distribution, which are often lacking in practical Bayesian problems. Our analysis hinges on statistical large sample theory, which constrains the deviation of the Bayesian posterior from being smooth and log-concave in a very specific way. In particular, we introduce a new technique for bounding the mixing time of a Markov chain with a continuous state space via the $s$-conductance profile, offering improvements over existing techniques in several aspects. By employing this new technique, we establish the optimal parameter dimension dependence of $d^{1/3}$ and condition number dependence of $\kappa$ in the non-asymptotic mixing time upper bound for MALA after the burn-in period, under a standard Bayesian setting where the target posterior distribution is close to a $d$-dimensional Gaussian distribution with a covariance matrix having a condition number $\kappa$. We also prove a matching mixing time lower bound for sampling from a multivariate Gaussian via MALA to complement the upper bound.