Studying the Impact of Augmentations on Medical Confidence Calibration

TL;DR

This study evaluates how modern image augmentation techniques like CutMix, MixUp, and CutOut affect the calibration and performance of CNNs in medical diagnosis, emphasizing the importance of confidence reliability.

Contribution

It provides a comparative analysis of modern augmentation methods on CNN calibration and performance specifically in medical imaging tasks.

Findings

CutMix significantly improves model calibration.

CutOut tends to decrease calibration accuracy.

Augmentation techniques influence both calibration and overall performance.

Abstract

The clinical explainability of convolutional neural networks (CNN) heavily relies on the joint interpretation of a model's predicted diagnostic label and associated confidence. A highly certain or uncertain model can significantly impact clinical decision-making. Thus, ensuring that confidence estimates reflect the true correctness likelihood for a prediction is essential. CNNs are often poorly calibrated and prone to overconfidence leading to improper measures of uncertainty. This creates the need for confidence calibration. However, accuracy and performance-based evaluations of CNNs are commonly used as the sole benchmark for medical tasks. Taking into consideration the risks associated with miscalibration is of high importance. In recent years, modern augmentation techniques, which cut, mix, and combine images, have been introduced. Such augmentations have benefited CNNs through regularization, robustness to adversarial samples, and calibration. Standard augmentations based on image scaling, rotating, and zooming, are widely leveraged in the medical domain to combat the scarcity of data. In this paper, we evaluate the effects of three modern augmentation techniques, CutMix, MixUp, and CutOut on the calibration and performance of CNNs for medical tasks. CutMix improved calibration the most while CutOut often lowered the level of calibration.