Affine invariant interacting Langevin dynamics for Bayesian inference

TL;DR

ALDI is a novel affine-invariant Langevin dynamics method for high-dimensional Bayesian sampling, utilizing an ensemble approach with covariance-based preconditioning, offering gradient-free options and strong theoretical guarantees.

Contribution

This paper introduces ALDI, a new affine-invariant Langevin sampling algorithm that is efficient for high-dimensional problems and does not require matrix inversions, with theoretical analysis and practical Bayesian applications.

Findings

ALDI is affine-invariant and suitable for high-dimensional sampling.

ALDI can operate without gradient information, useful in Bayesian inverse problems.

Theoretical properties like ergodicity are established for ALDI.

Abstract

We propose a computational method (with acronym ALDI) for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invariance. The central idea of ALDI is to run an ensemble of particles with their empirical covariance serving as a preconditioner for their underlying Langevin dynamics. ALDI does not require taking the inverse or square root of the empirical covariance matrix, which enables application to high-dimensional sampling problems. The theoretical properties of ALDI are studied in terms of non-degeneracy and ergodicity. Furthermore, we study its connections to diffusion on Riemannian manifolds and Wasserstein gradient flows. Bayesian inference serves as a main application area for ALDI. In case of a forward problem with additive Gaussian measurement errors, ALDI allows for a gradient-free approximation in the spirit of the ensemble Kalman filter. A computational comparison between gradient-free and gradient-based ALDI is provided for a PDE constrained Bayesian inverse problem.