Subspace Langevin Monte Carlo

TL;DR

Subspace Langevin Monte Carlo (SLMC) is a new sampling method that improves efficiency and adaptability for high-dimensional distributions by projecting Langevin updates onto eigenblocks, with proven error guarantees and practical success.

Contribution

We propose SLMC, a novel sampling algorithm that generalizes existing methods by using subspace projections, enhancing computational efficiency and adaptability in high-dimensional settings.

Findings

SLMC outperforms traditional Langevin methods in efficiency.

Error guarantees are established for SLMC.

Experiments demonstrate practical effectiveness on ill-conditioned distributions.

Abstract

Sampling from high-dimensional distributions has wide applications in data science and machine learning but poses significant computational challenges. We introduce Subspace Langevin Monte Carlo (SLMC), a novel and efficient sampling method that generalizes random-coordinate Langevin Monte Carlo and preconditioned Langevin Monte Carlo by projecting the Langevin update onto subsampled eigenblocks of a time-varying preconditioner at each iteration. The advantage of SLMC is its superior adaptability and computational efficiency compared to traditional Langevin Monte Carlo and preconditioned Langevin Monte Carlo. Using coupling arguments, we establish error guarantees for SLMC and demonstrate its practical effectiveness through a few experiments on sampling from ill-conditioned distributions.