Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions

TL;DR

This paper establishes fundamental lower bounds on the mixing times of MALA and HMC algorithms for well-conditioned distributions, revealing inherent limitations and the necessity of certain dependencies on dimension and condition number.

Contribution

It provides nearly-tight lower bounds on MALA's mixing time and shows polynomial dimension dependence for HMC, introducing a novel connection to Chebyshev polynomials.

Findings

Lower bound of (  d) on MALA's mixing time from warm start.

Polynomial dependence on dimension is necessary for HMC relaxation time.

Analysis introduces a new link between leapfrog integrator and Chebyshev polynomials.

Abstract

We give lower bounds on the performance of two of the most popular sampling methods in practice, the Metropolis-adjusted Langevin algorithm (MALA) and multi-step Hamiltonian Monte Carlo (HMC) with a leapfrog integrator, when applied to well-conditioned distributions. Our main result is a nearly-tight lower bound of $\widetilde{\Omega}(\kappa d)$ on the mixing time of MALA from an exponentially warm start, matching a line of algorithmic results up to logarithmic factors and answering an open question of Chewi et. al. We also show that a polynomial dependence on dimension is necessary for the relaxation time of HMC under any number of leapfrog steps, and bound the gains achievable by changing the step count. Our HMC analysis draws upon a novel connection between leapfrog integration and Chebyshev polynomials, which may be of independent interest.