A continuation method in Bayesian inference

TL;DR

This paper introduces a continuation method for Bayesian inference that smoothly transitions from prior to posterior distributions, enabling efficient sampling and analysis of complex models with non-Gaussian noise and multimodal parameters.

Contribution

The paper develops a novel continuation approach characterized by a nonlinear PDE, providing a stable computational framework for Bayesian inference without extensive forward model evaluations.

Findings

Demonstrates computational stability and efficiency through three numerical examples.

Shows effectiveness in handling non-Gaussian noise in Bayesian inference.

Validates the method's ability to invert multimodal parameters.

Abstract

We present a continuation method that entails generating a sequence of transition probability density functions from the prior to the posterior in the context of Bayesian inference for parameter estimation problems. The characterization of transition distributions, by tempering the likelihood function, results in a homogeneous nonlinear partial integro-differential equation whose existence and uniqueness of solutions are addressed. The posterior probability distribution comes as the interpretation of the final state of the path of transition distributions. A computationally stable scaling domain for the likelihood is explored for the approximation of the expected deviance, where we manage to hold back all the evaluations of the forward predictive model at the prior stage. It follows the computational tractability of the posterior distribution and opens access to the posterior distribution for direct samplings. To get a solution formulation of the expected deviance, we derive a partial differential equation governing the moments generating function of the log-likelihood. We show also that a spectral formulation of the expected deviance can be obtained for low-dimensional problems under certain conditions. The computational efficiency of the proposed method is demonstrated through three differents numerical examples that focus on analyzing the computational bias generated by the method, assessing the continuation method in the Bayesian inference with non-Gaussian noise, and evaluating its ability to invert a multimodal parameter of interest.