Less interaction with forward models in Langevin dynamics

TL;DR

This paper introduces adaptive ensemble enrichment strategies for Langevin dynamics to reduce forward model evaluations and improve convergence, especially for complex posterior distributions, demonstrated through numerical benchmarks.

Contribution

It proposes adaptive ensemble enrichment methods with analytical guarantees for linear models and extends them using homotopy-based Langevin dynamics for complex distributions.

Findings

Significant reduction in forward calls with enrichment strategies

Convergence guarantees for linear models

Improved sampling efficiency on benchmark problems

Abstract

Ensemble methods have become ubiquitous for the solution of Bayesian inference problems. State-of-the-art Langevin samplers such as the Ensemble Kalman Sampler (EKS), Affine Invariant Langevin Dynamics (ALDI) or its extension using weighted covariance estimates rely on successive evaluations of the forward model or its gradient. A main drawback of these methods hence is their vast number of required forward calls as well as their possible lack of convergence in the case of more involved posterior measures such as multimodal distributions. The goal of this paper is to address these challenges to some extend. First, several possible adaptive ensemble enrichment strategies that successively enlarge the number of particles in the underlying Langevin dynamics are discusses that in turn lead to a significant reduction of the total number of forward calls. Second, analytical consistency guarantees of the ensemble enrichment method are provided for linear forward models. Third, to address more involved target distributions, the method is extended by applying adapted Langevin dynamics based on a homotopy formalism for which convergence is proved. Finally, numerical investigations of several benchmark problems illustrates the possible gain of the proposed method, comparing it to state-of-the-art Langevin samplers.