Interval-Based AUC (iAUC): Extending ROC Analysis to Uncertainty-Aware Classification

TL;DR

This paper introduces an uncertainty-aware ROC framework for interval-valued predictions, with new measures $AUC_L$ and $AUC_U$, enabling better evaluation and decision-making in risk prediction tasks.

Contribution

It extends ROC analysis to interval predictions, providing bounds on optimal AUC and supporting selective abstention for uncertainty management.

Findings

$AUC_L$ and $AUC_U$ bounds on optimal AUC are validated.

Framework effectively supports uncertainty-aware evaluation.

Experimental results confirm practical utility on real datasets.

Abstract

In high-stakes risk prediction, quantifying uncertainty through interval-valued predictions is essential for reliable decision-making. However, standard evaluation tools like the receiver operating characteristic (ROC) curve and the area under the curve (AUC) are designed for point scores and fail to capture the impact of predictive uncertainty on ranking performance. We propose an uncertainty-aware ROC framework specifically for interval-valued predictions, introducing two new measures: $AUC_L$ and $AUC_U$. This framework enables an informative three-region decomposition of the ROC plane, partitioning pairwise rankings into correct, incorrect, and uncertain orderings. This approach naturally supports selective prediction by allowing models to abstain from ranking cases with overlapping intervals, thereby optimizing the trade-off between abstention rate and discriminative reliability. We prove that under valid class-conditional coverage, $AUC_L$ and $AUC_U$ provide formal lower and upper bounds on the theoretical optimal AUC ($AUC^*$), characterizing the physical limit of achievable discrimination. The proposed framework applies broadly to interval-valued prediction models, regardless of the interval construction method. Experiments on real-world benchmark datasets, using bootstrap-based intervals as one instantiation, validate the framework's correctness and demonstrate its practical utility for uncertainty-aware evaluation and decision-making.