BSM loss: A superior way in modeling aleatory uncertainty of fine_grained classification

TL;DR

This paper introduces a modified Bootstrapping loss with Mixup augmentation that improves uncertainty calibration in fine-grained medical image classification, enhancing reliability without extra inference cost.

Contribution

It proposes a novel BSM loss function combining Bootstrapping and Mixup to better model data-dependent uncertainty and calibrate confidence in low-data, noisy settings.

Findings

Halves Expected Calibration Error compared to other methods

Shows strong correlation between uncertainty and data similarity

Perceives semantic distance of out-of-domain data effectively

Abstract

Artificial intelligence(AI)-assisted method had received much attention in the risk field such as disease diagnosis. Different from the classification of disease types, it is a fine-grained task to classify the medical images as benign or malignant. However, most research only focuses on improving the diagnostic accuracy and ignores the evaluation of model reliability, which limits its clinical application. For clinical practice, calibration presents major challenges in the low-data regime extremely for over-parametrized models and inherent noises. In particular, we discovered that modeling data-dependent uncertainty is more conducive to confidence calibrations. Compared with test-time augmentation(TTA), we proposed a modified Bootstrapping loss(BS loss) function with Mixup data augmentation strategy that can better calibrate predictive uncertainty and capture data distribution transformation without additional inference time. Our experiments indicated that BS loss with Mixup(BSM) model can halve the Expected Calibration Error(ECE) compared to standard data augmentation, deep ensemble and MC dropout. The correlation between uncertainty and similarity of in-domain data is up to -0.4428 under the BSM model. Additionally, the BSM model is able to perceive the semantic distance of out-of-domain data, demonstrating high potential in real-world clinical practice.