Consensus Based Sampling

TL;DR

This paper introduces a derivative-free, affine invariant sampling and optimization method based on stochastic interacting particles, effective for inverse problems, Bayesian sampling, and finding MAP estimates, with proven convergence properties.

Contribution

The paper presents a novel consensus-based sampling and optimization method that is derivative-free and affine invariant, suitable for complex inverse problems and Bayesian inference.

Findings

Method effectively generates samples from target distributions.

Contraction properties are established under certain conditions.

Numerical experiments show wide basins of attraction and Laplace approximation recovery.

Abstract

We propose a novel method for sampling and optimization tasks based on a stochastic interacting particle system. We explain how this method can be used for the following two goals: (i) generating approximate samples from a given target distribution; (ii) optimizing a given objective function. The approach is derivative-free and affine invariant, and is therefore well-suited for solving inverse problems defined by complex forward models: (i) allows generation of samples from the Bayesian posterior and (ii) allows determination of the maximum a posteriori estimator. We investigate the properties of the proposed family of methods in terms of various parameter choices, both analytically and by means of numerical simulations. The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems. The analysis and experiments also demonstrate the potential for the sampling methodology in regimes in which the target distribution is unimodal and close to Gaussian; indeed we prove that the method recovers a Laplace approximation to the measure in certain parametric regimes and provide numerical evidence that this Laplace approximation attracts a large set of initial conditions in a number of examples.