CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference

TL;DR

This paper introduces CP4SBI, a conformal calibration framework that improves the accuracy of credible sets in simulation-based inference, ensuring better local coverage guarantees across various models and scoring functions.

Contribution

It presents a model-agnostic calibration method with two variants, enhancing the reliability of uncertainty quantification in SBI without assuming specific model forms.

Findings

Improves local coverage guarantees in SBI credible sets.

Enhances uncertainty quantification for neural posterior estimators.

Demonstrates effectiveness on standard SBI benchmarks.

Abstract

Current experimental scientists have been increasingly relying on simulation-based inference (SBI) to invert complex non-linear models with intractable likelihoods. However, posterior approximations obtained with SBI are often miscalibrated, causing credible regions to undercover true parameters. We develop $\texttt{CP4SBI}$, a model-agnostic conformal calibration framework that constructs credible sets with local Bayesian coverage. Our two proposed variants, namely local calibration via regression trees and CDF-based calibration, enable finite-sample local coverage guarantees for any scoring function, including HPD, symmetric, and quantile-based regions. Experiments on widely used SBI benchmarks demonstrate that our approach improves the quality of uncertainty quantification for neural posterior estimators using both normalizing flows and score-diffusion modeling.