Randomized maximum likelihood based posterior sampling

TL;DR

This paper introduces a novel approach for sampling from complex, multimodal high-dimensional posterior distributions using randomized maximum likelihood, improving sampling by considering all critical points and employing efficient approximations.

Contribution

The paper proposes a new method that enhances sampling from multimodal posteriors by computing all critical points and using low-rank approximations for efficiency in high dimensions.

Findings

Improved sampling by considering all critical points instead of only minimizers.

Efficient approximate weighting using low-rank Gauss-Newton approximation.

Successful application to toy problems and a Darcy flow problem with multiple modes.

Abstract

Minimization of a stochastic cost function is commonly used for approximate sampling in high-dimensional Bayesian inverse problems with Gaussian prior distributions and multimodal posterior distributions. The density of the samples generated by minimization is not the desired target density, unless the observation operator is linear, but the distribution of samples is useful as a proposal density for importance sampling or for Markov chain Monte Carlo methods. In this paper, we focus on applications to sampling from multimodal posterior distributions in high dimensions. We first show that sampling from multimodal distributions is improved by computing all critical points instead of only minimizers of the objective function. For applications to high-dimensional geoscience problems, we demonstrate an efficient approximate weighting that uses a low-rank Gauss-Newton approximation of the determinant of the Jacobian. The method is applied to two toy problems with known posterior distributions and a Darcy flow problem with multiple modes in the posterior.