A Critical Perspective on Finite Sample Conformal Prediction Theory in Medical Applications

TL;DR

This paper critically examines the practical limitations of conformal prediction in medical applications, highlighting that small calibration sets undermine the reliability of uncertainty estimates despite theoretical guarantees.

Contribution

It challenges the common assumption that conformal prediction guarantees are practically useful with small calibration samples in medical contexts.

Findings

Theoretical guarantees do not ensure practical utility with small calibration sets.

Empirical demonstration shows limited effectiveness of CP in medical image classification with scarce data.

Highlights the need for larger calibration sets for reliable uncertainty estimation in healthcare.

Abstract

Machine learning (ML) is transforming healthcare, but safe clinical decisions demand reliable uncertainty estimates that standard ML models fail to provide. Conformal prediction (CP) is a popular tool that allows users to turn heuristic uncertainty estimates into uncertainty estimates with statistical guarantees. CP works by converting predictions of a ML model, together with a calibration sample, into prediction sets that are guaranteed to contain the true label with any desired probability. An often cited advantage is that CP theory holds for calibration samples of arbitrary size, suggesting that uncertainty estimates with practically meaningful statistical guarantees can be achieved even if only small calibration sets are available. We question this promise by showing that, although the statistical guarantees hold for calibration sets of arbitrary size, the practical utility of these guarantees does highly depend on the size of the calibration set. This observation is relevant in medical domains because data is often scarce and obtaining large calibration sets is therefore infeasible. We corroborate our critique in an empirical demonstration on a medical image classification task.