Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictors

TL;DR

This paper broadens the scope of Adaptive Conformal Inference by demonstrating that confidence predictors, not just conformal predictors, can provide finite-sample coverage guarantees, offering computational and efficiency advantages.

Contribution

It introduces confidence predictors as a valid alternative to conformal predictors for ACI, expanding theoretical guarantees and practical benefits in non-exchangeable data scenarios.

Findings

NCCP maintains coverage guarantees similar to CP.

NCCP offers computational advantages in online settings.

INCCP outperforms ICP in limited data scenarios.

Abstract

Adaptive Conformal Inference (ACI) provides finite-sample coverage guarantees, enhancing the prediction reliability under non-exchangeability. This study demonstrates that these desirable properties of ACI do not require the use of Conformal Predictors (CP). We show that the guarantees hold for the broader class of confidence predictors, defined by the requirement of producing nested prediction sets, a property we argue is essential for meaningful confidence statements. We empirically investigate the performance of Non-Conformal Confidence Predictors (NCCP) against CP when used with ACI on non-exchangeable data. In online settings, the NCCP offers significant computational advantages while maintaining a comparable predictive efficiency. In batch settings, inductive NCCP (INCCP) can outperform inductive CP (ICP) by utilising the full training dataset without requiring a separate calibration set, leading to improved efficiency, particularly when the data are limited. Although these initial results highlight NCCP as a theoretically sound and practically effective alternative to CP for uncertainty quantification with ACI in non-exchangeable scenarios, further empirical studies are warranted across diverse datasets and predictors.