Propagating Surrogate Uncertainty in Bayesian Inverse Problems

TL;DR

This paper introduces a new framework for propagating surrogate model uncertainty in Bayesian inverse problems, compares existing heuristics, and proposes a novel MCMC algorithm for practical uncertainty quantification.

Contribution

It proposes the expected posterior (EP) as a general baseline for uncertainty-aware Bayesian inference and introduces the RKpCN algorithm for efficient sampling with Gaussian process surrogates.

Findings

The heuristic EUP can fail with non-uniform surrogate uncertainty.

EP provides a more reliable uncertainty propagation baseline.

RKpCN enables practical Bayesian inference with complex surrogates.

Abstract

Standard Bayesian inference schemes are infeasible for inverse problems with computationally expensive forward models. A common solution is to replace the model with a cheaper surrogate. To avoid overconfident conclusions, it is essential to acknowledge the surrogate approximation by propagating its uncertainty. At present, a variety of distinct uncertainty propagation methods have been suggested, with little understanding of how they vary. To fill this gap, we propose a mixture distribution termed the expected posterior (EP) as a general baseline for uncertainty-aware posterior approximation, justified by decision theoretic and modular Bayesian inference arguments. We then investigate the expected unnormalized posterior (EUP), a popular heuristic alternative, analyzing when it may deviate from the EP baseline. Our results show that this heuristic can break down when the surrogate uncertainty is highly non-uniform over the design space, as can be the case when the log-likelihood is emulated by a Gaussian process. Finally, we present the random kernel preconditioned Crank-Nicolson (RKpCN) algorithm, an approximate Markov chain Monte Carlo scheme that provides practical EP approximation in the challenging setting involving infinite-dimensional Gaussian process surrogates.