Bayesian Conformal Prediction as a Decision Risk Problem

TL;DR

This paper introduces Bayesian Conformal Prediction (BCP), a framework combining Bayesian distributions with conformal risk control to produce efficient, valid prediction sets, especially effective with multimodal distributions.

Contribution

BCP formulates conformal prediction as a decision-risk optimization, enabling the creation of disjoint, high-density prediction sets with finite-sample coverage guarantees.

Findings

HPD sets improve efficiency in multimodal distributions.

BCP maintains coverage under model misspecification.

Reduces mean prediction set size from 4.82 to 2.07 in experiments.

Abstract

We propose Bayesian Conformal Prediction (BCP), a framework that combines Bayesian posterior predictive distributions with PAC-style conformal risk control to produce prediction sets with finite-sample coverage guarantees. Standard quantile-threshold conformal methods often construct prediction sets using a single fixed threshold, which typically yields connected prediction sets. While valid, such sets can be inefficient when the posterior predictive distribution is multimodal, since they may span low-density regions between separated modes. The main contribution of BCP is to formulate conformal prediction as a decision-risk optimisation problem, extending standard fixed quantile-threshold sets to optimised highest posterior density (HPD) prediction sets. These sets can be disjoint, concentrating probability mass on separated high-density regions. Validity is enforced using a PAC-style risk constraint, which provides coverage control even when the Bayesian model is misspecified. In standard nested-threshold settings, BCP recovers the smallest feasible threshold, aligning with existing PAC-based approaches. In the multimodal experiment, HPD geometry substantially improves efficiency, reducing mean prediction set size from $4.82$ to $2.07$ while satisfying the target PAC pass rate. Across regression, classification, and distribution-shift experiments, BCP maintains reliable coverage under model misspecification, whereas Bayesian credible intervals can fail to preserve nominal coverage.