Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage

TL;DR

Kandinsky conformal prediction extends coverage guarantees beyond traditional methods by providing flexible, overlapping, and fractional group conditional coverage, unifying and improving upon existing conformal prediction techniques with practical empirical validation.

Contribution

The paper introduces Kandinsky conformal prediction, a novel framework that offers flexible, overlapping, and fractional group coverage guarantees, unifying and extending prior conformal methods.

Findings

Achieves minimax-optimal high-probability conditional coverage bounds.

Unifies covariate-based, class-conditional, and Mondrian conformal prediction.

Demonstrates practical effectiveness on real-world datasets.

Abstract

Conformal prediction is a powerful distribution-free framework for constructing prediction sets with coverage guarantees. Classical methods, such as split conformal prediction, provide marginal coverage, ensuring that the prediction set contains the label of a random test point with a target probability. However, these guarantees may not hold uniformly across different subpopulations, leading to disparities in coverage. Prior work has explored coverage guarantees conditioned on events related to the covariates and label of the test point. We present Kandinsky conformal prediction, a framework that significantly expands the scope of conditional coverage guarantees. In contrast to Mondrian conformal prediction, which restricts its coverage guarantees to disjoint groups -- reminiscent of the rigid, structured grids of Piet Mondrian's art -- our framework flexibly handles overlapping and fractional group memberships defined jointly on covariates and labels, reflecting the layered, intersecting forms in Wassily Kandinsky's compositions. Our algorithm unifies and extends existing methods, encompassing covariate-based group conditional, class conditional, and Mondrian conformal prediction as special cases, while achieving a minimax-optimal high-probability conditional coverage bound. Finally, we demonstrate the practicality of our approach through empirical evaluation on real-world datasets.