Geometrically adapted Langevin dynamics for Markov chain Monte Carlo simulations

TL;DR

This paper introduces a geometrically adapted Langevin dynamics method for MCMC sampling that leverages differential geometry to improve performance in high-dimensional and anisotropic problems, outperforming existing algorithms.

Contribution

The paper develops a Riemannian manifold-based Langevin dynamics approach, enhancing sampling efficiency over traditional MALA and HMC, especially in complex, high-dimensional spaces.

Findings

GALA outperforms MALA and HMC in high-dimensional problems

The method is effective for parameter estimation in complex distributions

GALA is often the only successful method in challenging scenarios

Abstract

Markov Chain Monte Carlo (MCMC) is one of the most powerful methods to sample from a given probability distribution, of which the Metropolis Adjusted Langevin Algorithm (MALA) is a variant wherein the gradient of the distribution is used towards faster convergence. However, being set up in the Euclidean framework, MALA might perform poorly in higher dimensional problems or in those involving anisotropic densities as the underlying non-Euclidean aspects of the geometry of the sample space remain unaccounted for. We make use of concepts from differential geometry and stochastic calculus on Riemannian manifolds to geometrically adapt a stochastic differential equation with a non-trivial drift term. This adaptation is also referred to as a stochastic development. We apply this method specifically to the Langevin diffusion equation and arrive at a geometrically adapted Langevin dynamics. This new approach far outperforms MALA, certain manifold variants of MALA, and other approaches such as Hamiltonian Monte Carlo (HMC), its adaptive variant the no-U-turn sampler (NUTS) implemented in Stan, especially as the dimension of the problem increases where often GALA is actually the only successful method. This is evidenced through several numerical examples that include parameter estimation of a broad class of probability distributions and a logistic regression problem.