Accelerating Langevin Monte Carlo via Efficient Stochastic Runge--Kutta Methods beyond Log-Concavity

TL;DR

This paper introduces a more efficient higher-order Langevin Monte Carlo algorithm using stochastic Runge--Kutta methods, achieving faster convergence and requiring fewer gradient evaluations in high-dimensional sampling tasks.

Contribution

The paper proposes a novel Hessian-free, higher-order LMC algorithm based on stochastic Runge--Kutta methods with improved computational efficiency and convergence analysis beyond log-concavity.

Findings

Achieves a uniform-in-time convergence rate of O(d^{3/2} h^{3/2}) in non-log-concave settings.

Requires only two gradient evaluations per iteration, reducing computational cost.

Demonstrates effectiveness through numerical experiments.

Abstract

Sampling from a high-dimensional probability distribution is a fundamental algorithmic task arising in wide-ranging applications across multiple disciplines, including scientific computing, computational statistics and machine learning. Langevin Monte Carlo (LMC) algorithms are among the most widely used sampling methods in high-dimensional settings. This paper introduces a novel higher-order and Hessian-free LMC sampling algorithm based on an efficient stochastic Runge--Kutta method of strong order $1.5$ for the overdamped Langevin dynamics. In contrast to the existing Runge--Kutta type LMC (Li et al., 2019) involved with three gradient evaluations, the newly proposed algorithm is computationally cheaper and requires only two gradient evaluations for one iteration. Under certain log-smooth conditions, non-asymptotic error bounds of the proposed algorithms are analyzed in $\mathcal{W}_2$-distance. In particular, a uniform-in-time convergence rate of order $O(d ^{\frac32} h^{\frac32})$ is derived in a non-log-concave setting, matching the convergence rate proved in the aforementioned work but under the log-concavity condition. Numerical experiments are finally presented to demonstrate the effectiveness of the new sampling algorithm.