Conformal Prediction with Learned Features

TL;DR

This paper introduces Partition Learning Conformal Prediction (PLCP), a novel framework that learns uncertainty-guided features to improve the conditional validity of prediction sets, demonstrating superior performance over existing methods.

Contribution

The paper proposes PLCP, a new method that learns features to enhance conditional coverage guarantees in conformal prediction, with theoretical analysis and efficient implementation.

Findings

PLCP achieves better coverage and shorter prediction sets than state-of-the-art methods.

PLCP provides theoretical conditional guarantees for both finite and infinite samples.

Experimental results validate PLCP's effectiveness across multiple datasets.

Abstract

In this paper, we focus on the problem of conformal prediction with conditional guarantees. Prior work has shown that it is impossible to construct nontrivial prediction sets with full conditional coverage guarantees. A wealth of research has considered relaxations of full conditional guarantees, relying on some predefined uncertainty structures. Departing from this line of thinking, we propose Partition Learning Conformal Prediction (PLCP), a framework to improve conditional validity of prediction sets through learning uncertainty-guided features from the calibration data. We implement PLCP efficiently with alternating gradient descent, utilizing off-the-shelf machine learning models. We further analyze PLCP theoretically and provide conditional guarantees for infinite and finite sample sizes. Finally, our experimental results over four real-world and synthetic datasets show the superior performance of PLCP compared to state-of-the-art methods in terms of coverage and length in both classification and regression scenarios.