A Conformal Prediction Score that is Robust to Label Noise

TL;DR

This paper introduces a robust conformal prediction score that effectively handles label noise in calibration data, improving uncertainty quantification in medical imaging classification.

Contribution

The authors propose a novel conformal score estimation method that accounts for label noise, enhancing the reliability of prediction sets in noisy label scenarios.

Findings

Outperforms existing methods in prediction set size

Maintains coverage despite noisy labels

Effective on medical imaging datasets

Abstract

Conformal Prediction (CP) quantifies network uncertainty by building a small prediction set with a pre-defined probability that the correct class is within this set. In this study we tackle the problem of CP calibration based on a validation set with noisy labels. We introduce a conformal score that is robust to label noise. The noise-free conformal score is estimated using the noisy labeled data and the noise level. In the test phase the noise-free score is used to form the prediction set. We applied the proposed algorithm to several standard medical imaging classification datasets. We show that our method outperforms current methods by a large margin, in terms of the average size of the prediction set, while maintaining the required coverage.