Conformal Bayesian Computation

TL;DR

This paper introduces scalable conformal Bayesian methods that produce predictive intervals with finite-sample guarantees, combining Bayesian posterior distributions with conformal inference to ensure reliable coverage even under model misspecification.

Contribution

The authors develop an efficient 'add-one-in' importance sampling approach to compute conformal Bayesian predictive intervals without costly refitting or data splitting.

Findings

Efficient computation of conformal Bayesian intervals from re-weighted posterior samples.

Finite-sample coverage guarantees under model misspecification.

Extension to hierarchical and partially exchangeable models.

Abstract

We develop scalable methods for producing conformal Bayesian predictive intervals with finite sample calibration guarantees. Bayesian posterior predictive distributions, $p(y \mid x)$, characterize subjective beliefs on outcomes of interest, $y$, conditional on predictors, $x$. Bayesian prediction is well-calibrated when the model is true, but the predictive intervals may exhibit poor empirical coverage when the model is misspecified, under the so called ${\cal{M}}$-open perspective. In contrast, conformal inference provides finite sample frequentist guarantees on predictive confidence intervals without the requirement of model fidelity. Using 'add-one-in' importance sampling, we show that conformal Bayesian predictive intervals are efficiently obtained from re-weighted posterior samples of model parameters. Our approach contrasts with existing conformal methods that require expensive refitting of models or data-splitting to achieve computational efficiency. We demonstrate the utility on a range of examples including extensions to partially exchangeable settings such as hierarchical models.