High-accuracy sampling from constrained spaces with the Metropolis-adjusted Preconditioned Langevin Algorithm

TL;DR

This paper introduces a new first-order sampling algorithm, the Metropolis-adjusted Preconditioned Langevin Algorithm, which efficiently samples from convex constrained spaces with high accuracy, supported by theoretical mixing time bounds and practical experiments.

Contribution

The paper develops a novel sampling method combining Metropolis-Hastings with preconditioned Langevin dynamics, providing non-asymptotic mixing bounds and demonstrating improved dependence on dimension under stronger conditions.

Findings

The proposed method achieves high-accuracy sampling with polylogarithmic error dependence.

Non-asymptotic mixing time bounds are derived for convex and exponential distributions.

Numerical experiments confirm the practicality and efficiency of the algorithm.

Abstract

In this work, we propose a first-order sampling method called the Metropolis-adjusted Preconditioned Langevin Algorithm for approximate sampling from a target distribution whose support is a proper convex subset of $\mathbb{R}^{d}$. Our proposed method is the result of applying a Metropolis-Hastings filter to the Markov chain formed by a single step of the preconditioned Langevin algorithm with a metric $\mathscr{G}$, and is motivated by the natural gradient descent algorithm for optimisation. We derive non-asymptotic upper bounds for the mixing time of this method for sampling from target distributions whose potentials are bounded relative to $\mathscr{G}$, and for exponential distributions restricted to the support. Our analysis suggests that if $\mathscr{G}$ satisfies stronger notions of self-concordance introduced in Kook and Vempala (2024), then these mixing time upper bounds have a strictly better dependence on the dimension than when is merely self-concordant. We also provide numerical experiments that demonstrates the practicality of our proposed method. Our method is a high-accuracy sampler due to the polylogarithmic dependence on the error tolerance in our mixing time upper bounds.