Dimension-Robust MCMC in Bayesian Inverse Problems

TL;DR

This paper introduces a dimension-robust MCMC framework for Bayesian inverse problems, enabling efficient sampling in high-dimensional, non-Gaussian, and hierarchical models, with applications in classification and uncertainty quantification.

Contribution

It develops a novel MCMC framework combining non-centred parameterisations and dimension-robust samplers, effective for high-dimensional Bayesian inverse problems with complex priors.

Findings

MCMC efficiency remains stable as dimension increases.

Framework successfully applied to semi-supervised multi-class classification.

Active learning with the sampler reduces the need for labeled data.

Abstract

The methodology developed in this article is motivated by a wide range of prediction and uncertainty quantification problems that arise in Statistics, Machine Learning and Applied Mathematics, such as non-parametric regression, multi-class classification and inversion of partial differential equations. One popular formulation of such problems is as Bayesian inverse problems, where a prior distribution is used to regularize inference on a high-dimensional latent state, typically a function or a field. It is common that such priors are non-Gaussian, for example piecewise-constant or heavy-tailed, and/or hierarchical, in the sense of involving a further set of low-dimensional parameters, which, for example, control the scale or smoothness of the latent state. In this formulation prediction and uncertainty quantification relies on efficient exploration of the posterior distribution of latent states and parameters. This article introduces a framework for efficient MCMC sampling in Bayesian inverse problems that capitalizes upon two fundamental ideas in MCMC, non-centred parameterisations of hierarchical models and dimension-robust samplers for latent Gaussian processes. Using a range of diverse applications we showcase that the proposed framework is dimension-robust, that is, the efficiency of the MCMC sampling does not deteriorate as the dimension of the latent state gets higher. We showcase the full potential of the machinery we develop in the article in semi-supervised multi-class classification, where our sampling algorithm is used within an active learning framework to guide the selection of input data to manually label in order to achieve high predictive accuracy with a minimal number of labelled data.