Conformal Prediction via Transported Beta Laws

TL;DR

This paper introduces a novel framework for analyzing conformal prediction calibration by modeling the conditional coverage law as a Beta distribution and quantifying deviations using Wasserstein distances, improving understanding of non-i.i.d. data effects.

Contribution

It develops a new finite-sample analysis method for conformal prediction that captures non-i.i.d. behavior through Beta law deformations and Wasserstein metrics.

Findings

Beta law accurately models calibration-conditional coverage in i.i.d. setting.

Wasserstein distances quantify deviations from the Beta reference under non-i.i.d. conditions.

Simulations show the approximation remains accurate at moderate sample sizes.

Abstract

Split conformal prediction provides finite-sample marginal coverage under exchangeability, but this guarantee averages over the random calibration sample. We study instead the law of the calibration-conditional coverage induced by a realized conformal threshold. In the continuous i.i.d. setting this law is exactly $Beta(k,n+1-k)$, so the usual marginal guarantee corresponds to its mean. We take this beta law as a finite-sample reference object and quantify departures from it using Wasserstein distances on $[0,1]$. The framework yields direct bounds on marginal coverage gaps and on bad-calibration probabilities, and separates different sources of non-i.i.d. behavior according to how they deform the beta reference: test-side shift acts through a transport map on the coverage scale, while calibration dependence changes the order-statistic law itself. We instantiate the framework in scale-shift, clustered, and stationary mixing settings, where the induced deformations can be characterized explicitly or through Berry-Esseen approximations. Simulations on dependent processes confirm that the first-order approximation tracks the empirical Wasserstein distance even at moderate sample sizes.