Improving Deep Learning Model Calibration for Cardiac Applications using Deterministic Uncertainty Networks and Uncertainty-aware Training

TL;DR

This paper evaluates deterministic uncertainty methods and uncertainty-aware training to improve deep learning model calibration in cardiac imaging, demonstrating enhanced accuracy and calibration, and proposing a novel combined approach.

Contribution

It introduces a combined deterministic uncertainty-aware training method that improves calibration and accuracy in cardiac imaging applications.

Findings

Both DUMs and uncertainty-aware training improve calibration and accuracy.

DUMs generally outperform other methods in improvements.

The combined approach yields further benefits in some cases.

Abstract

Improving calibration performance in deep learning (DL) classification models is important when planning the use of DL in a decision-support setting. In such a scenario, a confident wrong prediction could lead to a lack of trust and/or harm in a high-risk application. We evaluate the impact on accuracy and calibration of two types of approach that aim to improve DL classification model calibration: deterministic uncertainty methods (DUM) and uncertainty-aware training. Specifically, we test the performance of three DUMs and two uncertainty-aware training approaches as well as their combinations. To evaluate their utility, we use two realistic clinical applications from the field of cardiac imaging: artefact detection from phase contrast cardiac magnetic resonance (CMR) and disease diagnosis from the public ACDC CMR dataset. Our results indicate that both DUMs and uncertainty-aware training can improve both accuracy and calibration in both of our applications, with DUMs generally offering the best improvements. We also investigate the combination of the two approaches, resulting in a novel deterministic uncertainty-aware training approach. This provides further improvements for some combinations of DUMs and uncertainty-aware training approaches.