Effect of Data Augmentation on Conformal Prediction for Diabetic Retinopathy

TL;DR

This study examines how different data augmentation techniques influence the reliability and efficiency of conformal prediction in diabetic retinopathy grading, emphasizing the importance of co-designing augmentation with uncertainty quantification.

Contribution

It systematically evaluates the impact of various data augmentation methods on conformal prediction performance in medical imaging, revealing that sample-mixing strategies enhance reliability.

Findings

Mixup and CutMix improve predictive accuracy and uncertainty reliability.

CLAHE can negatively affect model certainty.

Co-designing augmentation with uncertainty quantification is crucial.

Abstract

The clinical deployment of deep learning models for high-stakes tasks such as diabetic retinopathy (DR) grading requires demonstrable reliability. While models achieve high accuracy, their clinical utility is limited by a lack of robust uncertainty quantification. Conformal prediction (CP) offers a distribution-free framework to generate prediction sets with statistical guarantees of coverage. However, the interaction between standard training practices like data augmentation and the validity of these guarantees is not well understood. In this study, we systematically investigate how different data augmentation strategies affect the performance of conformal predictors for DR grading. Using the DDR dataset, we evaluate two backbone architectures -- ResNet-50 and a Co-Scale Conv-Attentional Transformer (CoaT) -- trained under five augmentation regimes: no augmentation, standard geometric transforms, CLAHE, Mixup, and CutMix. We analyze the downstream effects on conformal metrics, including empirical coverage, average prediction set size, and correct efficiency. Our results demonstrate that sample-mixing strategies like Mixup and CutMix not only improve predictive accuracy but also yield more reliable and efficient uncertainty estimates. Conversely, methods like CLAHE can negatively impact model certainty. These findings highlight the need to co-design augmentation strategies with downstream uncertainty quantification in mind to build genuinely trustworthy AI systems for medical imaging.