A conformalized learning of a prediction set with applications to medical imaging classification

TL;DR

This paper introduces a conformalized learning algorithm that modifies classifiers to produce reliable prediction sets with specified confidence levels, specifically applied to medical imaging classification.

Contribution

It presents a novel method to produce confidence-calibrated prediction sets for classifiers, ensuring desired coverage with smaller prediction sets in medical imaging tasks.

Findings

Outperforms existing methods with smaller prediction sets

Maintains high coverage probability

Effective across multiple medical imaging datasets

Abstract

Medical imaging classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, which prevents their deployment in medical clinics. We present an algorithm that can modify any classifier to produce a prediction set containing the true label with a user-specified probability, such as 90%. We train a network to predict an instance-based version of the Conformal Prediction threshold. The threshold is then conformalized to ensure the required coverage. We applied the proposed algorithm to several standard medical imaging classification datasets. The experimental results demonstrate that our method outperforms current approaches in terms of smaller average size of the prediction set while maintaining the desired coverage.