Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification

TL;DR

This paper introduces a model-agnostic conformal prediction method for ordinal classification that produces minimal-length, instance-specific prediction sets with guaranteed coverage, improving efficiency over existing approaches.

Contribution

It formulates ordinal conformal prediction as a minimum-length covering problem and develops a linear-time sliding-window algorithm for optimal prediction intervals.

Findings

Achieves 15% average reduction in prediction set size across datasets.

Provides instance-level optimal prediction intervals with coverage guarantees.

Demonstrates superior efficiency compared to baseline methods.

Abstract

Ordinal classification has been widely applied in many high-stakes applications, e.g., medical imaging and diagnosis, where reliable uncertainty quantification (UQ) is essential for decision making. Conformal prediction (CP) is a general UQ framework that provides statistically valid guarantees, which is especially useful in practice. However, prior ordinal CP methods mainly focus on heuristic algorithms or restrictively require the underlying model to predict a unimodal distribution over ordinal labels. Consequently, they provide limited insight into coverage-efficiency trade-offs, or a model-agnostic and distribution-free nature favored by CP methods. To this end, we fill this gap by propose an ordinal-CP method that is model-agnostic and provides instance-level optimal prediction intervals. Specifically, we formulate conformal ordinal classification as a minimum-length covering problem at the instance level. To solve this problem, we develop a sliding-window algorithm that is optimal on each calibration data, with only a linear time complexity in K, the number of label candidates. The local optimality per instance further also improves predictive efficiency in expectation. Moreover, we propose a length-regularized variant that shrinks prediction set size while preserving coverage. Experiments on four benchmark datasets from diverse domains are conducted to demonstrate the significantly improved predictive efficiency of the proposed methods over baselines (by 15% decrease on average over four datasets).