Integrating Uncertainty Awareness into Conformalized Quantile Regression

TL;DR

This paper introduces Uncertainty-Aware CQR (UACQR), a novel method that separates aleatoric and epistemic uncertainties in prediction intervals, improving coverage in diverse regions without distributional assumptions.

Contribution

The paper proposes UACQR, an extension of CQR that explicitly distinguishes and adjusts for different uncertainty sources, enhancing conditional coverage guarantees.

Findings

UACQR achieves stronger conditional coverage in experiments.

The method maintains distribution-free coverage guarantees.

UACQR outperforms traditional CQR in simulated and real-world data.

Abstract

Conformalized Quantile Regression (CQR) is a recently proposed method for constructing prediction intervals for a response $Y$ given covariates $X$, without making distributional assumptions. However, existing constructions of CQR can be ineffective for problems where the quantile regressors perform better in certain parts of the feature space than others. The reason is that the prediction intervals of CQR do not distinguish between two forms of uncertainty: first, the variability of the conditional distribution of $Y$ given $X$ (i.e., aleatoric uncertainty), and second, our uncertainty in estimating this conditional distribution (i.e., epistemic uncertainty). This can lead to intervals that are overly narrow in regions where epistemic uncertainty is high. To address this, we propose a new variant of the CQR methodology, Uncertainty-Aware CQR (UACQR), that explicitly separates these two sources of uncertainty to adjust quantile regressors differentially across the feature space. Compared to CQR, our methods enjoy the same distribution-free theoretical coverage guarantees, while demonstrating in our experiments stronger conditional coverage properties in simulated settings and real-world data sets alike.