Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation

TL;DR

This paper investigates the calibration of deep neural networks for medical image segmentation, comparing loss functions, proposing ensembling for better confidence estimates, and evaluating out-of-distribution detection across multiple medical imaging tasks.

Contribution

It introduces a systematic comparison of loss functions, proposes model ensembling for confidence calibration, and evaluates uncertainty estimation and out-of-distribution detection in medical segmentation.

Findings

Ensembling improves confidence calibration.

Dice loss affects uncertainty estimation differently than cross entropy.

Calibrated models better predict segmentation quality and detect OOD examples.

Abstract

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions both in correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.