RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classification

TL;DR

This paper introduces RR-CP, a conformal prediction method that guarantees a specified error rate in medical image classification, resulting in reliable and smaller prediction sets for clinical decision support.

Contribution

RR-CP is a novel conformal prediction approach that provides stronger statistical guarantees and optimized prediction set sizes under error constraints.

Findings

RR-CP achieves the user-specified error rate more frequently than existing methods.

RR-CP produces smaller prediction sets while maintaining reliability.

Experiments on five datasets validate the effectiveness of RR-CP.

Abstract

Conformal prediction (CP) generates a set of predictions for a given test sample such that the prediction set almost always contains the true label (e.g., 99.5\% of the time). CP provides comprehensive predictions on possible labels of a given test sample, and the size of the set indicates how certain the predictions are (e.g., a set larger than one is `uncertain'). Such distinct properties of CP enable effective collaborations between human experts and medical AI models, allowing efficient intervention and quality check in clinical decision-making. In this paper, we propose a new method called Reliable-Region-Based Conformal Prediction (RR-CP), which aims to impose a stronger statistical guarantee so that the user-specified error rate (e.g., 0.5\%) can be achieved in the test time, and under this constraint, the size of the prediction set is optimized (to be small). We consider a small prediction set size an important measure only when the user-specified error rate is achieved. Experiments on five public datasets show that our RR-CP performs well: with a reasonably small-sized prediction set, it achieves the user-specified error rate (e.g., 0.5\%) significantly more frequently than exiting CP methods.