Adjusting Regression Models for Conditional Uncertainty Calibration

TL;DR

This paper introduces a new algorithm to enhance the conditional coverage of conformal prediction in regression models, aiming to provide more reliable uncertainty estimates for high-stakes decisions.

Contribution

It proposes an end-to-end training method that reduces the gap between actual and nominal coverage, improving the reliability of conformal prediction methods.

Findings

Empirical results show improved conditional coverage on synthetic datasets.

Method effectively reduces miscoverage gap in real-world data.

Algorithm provides finite-sample guarantees for conditional coverage.

Abstract

Conformal Prediction methods have finite-sample distribution-free marginal coverage guarantees. However, they generally do not offer conditional coverage guarantees, which can be important for high-stakes decisions. In this paper, we propose a novel algorithm to train a regression function to improve the conditional coverage after applying the split conformal prediction procedure. We establish an upper bound for the miscoverage gap between the conditional coverage and the nominal coverage rate and propose an end-to-end algorithm to control this upper bound. We demonstrate the efficacy of our method empirically on synthetic and real-world datasets.