Fast and Efficient Parallel Sampling Using Higher Order Langevin Dynamics

TL;DR

This paper introduces a higher-order Langevin dynamics method that enables fast, parallel sampling from high-dimensional log-concave distributions with reduced resource requirements.

Contribution

It combines arbitrary-order Langevin dynamics with polynomial interpolation to lower parallel resource needs while maintaining polylogarithmic sequential depth.

Findings

Reduces parallel resource burden compared to existing methods.

Achieves target accuracy with fewer parallel points due to sharper discretization.

Applicable to models like Bayesian logistic regression and neural networks.

Abstract

We study parallel sampling from high-dimensional strongly log-concave distributions. Langevin-based samplers converge rapidly in continuous time, but their discretizations are typically sequential and often require polynomially many steps in the dimension $d$, the target accuracy $\varepsilon^{-1}$, or both. Picard-based parallel sampling methods reduce this sequential depth to polylogarithmic scale by solving for many time-discretization points in parallel; however, existing guarantees often require a polynomial number of processors, leading to substantial memory and gradient-evaluation costs in high dimensions. We show that higher-order Langevin structure can reduce this parallel resource burden while preserving polylogarithmic sequential depth. Our method combines arbitrary-order Langevin dynamics with blockwise Lagrange polynomial interpolation. This sharper discretization reduces the number of parallel points required to achieve a target accuracy. Our results cover both higher-order smooth potentials and ridge-separable potentials, including models such as Bayesian logistic regression and two-layer neural networks, and improve upon the space complexity of the current literature on parallel log-concave sampling.