Probabilistic Conformal Prediction with Approximate Conditional Validity

TL;DR

This paper introduces a new probabilistic conformal prediction method that achieves approximate conditional coverage by leveraging estimates of the conditional distribution, improving reliability in heteroscedastic settings.

Contribution

It extends existing conformal methods to provide approximately conditional coverage with theoretical bounds, enhancing prediction set adaptivity and reliability.

Findings

Outperforms existing methods in conditional coverage

Provides non-asymptotic bounds based on total variation distance

Demonstrates effectiveness through extensive simulations

Abstract

We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution $P_{Y \mid X}$. Existing methods, such as conformalized quantile regression and probabilistic conformal prediction, usually provide only a marginal coverage guarantee. In contrast, our approach extends these frameworks to achieve approximately conditional coverage, which is crucial for many practical applications. Our prediction sets adapt to the behavior of the predictive distribution, making them effective even under high heteroscedasticity. While exact conditional guarantees are infeasible without assumptions on the underlying data distribution, we derive non-asymptotic bounds that depend on the total variation distance of the conditional distribution and its estimate. Using extensive simulations, we show that our method consistently outperforms existing approaches in terms of conditional coverage, leading to more reliable statistical inference in a variety of applications.