Detecting Model Misspecification in Bayesian Inverse Problems via Variational Gradient Descent

TL;DR

This paper introduces a method to detect model misspecification in Bayesian inverse problems by comparing standard Bayesian posteriors with predictive-oriented posteriors, using an efficient variational gradient descent algorithm.

Contribution

The authors propose a novel empirical approach to identify model misspecification by contrasting Bayesian and predictive-oriented posteriors, supported by a new variational gradient descent algorithm.

Findings

Model misspecification can be detected by comparing posteriors.

The proposed algorithm is efficient and scalable.

Case studies confirm the method's effectiveness.

Abstract

Bayesian inference is optimal when the statistical model is well-specified, while outside this setting Bayesian inference can catastrophically fail; accordingly a wealth of post-Bayesian methodologies have been proposed. Predictively oriented (PrO) approaches lift the statistical model $P_\theta$ to an (infinite) mixture model $\int P_\theta \; \mathrm{d}Q(\theta)$ and fit this predictive distribution via minimising an entropy-regularised objective functional. In the well-specified setting one expects the mixing distribution $Q$ to concentrate around the true data-generating parameter in the large data limit, while such singular concentration will typically not be observed if the model is misspecified. Our contribution is to demonstrate that one can empirically detect model misspecification by comparing the standard Bayesian posterior to the PrO `posterior' $Q$. To operationalise this, we present an efficient numerical algorithm based on variational gradient descent. A simulation study, and a more detailed case study involving a Bayesian inverse problem in seismology, confirm that model misspecification can be automatically detected using this framework.