Informed Deep Abstaining Classifier: Investigating noise-robust training for diagnostic decision support systems

TL;DR

This paper introduces the Informed Deep Abstaining Classifier (IDAC), a noise-robust training method for diagnostic decision support systems that leverages noise level estimations to improve accuracy on noisy radiological data.

Contribution

The study extends the Deep Abstaining Classifier (DAC) loss to incorporate noise estimations, enhancing robustness in training with noisy clinical data.

Findings

IDAC outperforms DAC and other state-of-the-art losses in noisy conditions.

Results validated on public and in-house noisy chest X-ray datasets.

IDAC improves reliability of deep learning-based diagnostic systems.

Abstract

Image-based diagnostic decision support systems (DDSS) utilizing deep learning have the potential to optimize clinical workflows. However, developing DDSS requires extensive datasets with expert annotations and is therefore costly. Leveraging report contents from radiological data bases with Natural Language Processing to annotate the corresponding image data promises to replace labor-intensive manual annotation. As mining "real world" databases can introduce label noise, noise-robust training losses are of great interest. However, current noise-robust losses do not consider noise estimations that can for example be derived based on the performance of the automatic label generator used. In this study, we expand the noise-robust Deep Abstaining Classifier (DAC) loss to an Informed Deep Abstaining Classifier (IDAC) loss by incorporating noise level estimations during training. Our findings demonstrate that IDAC enhances the noise robustness compared to DAC and several state-of-the-art loss functions. The results are obtained on various simulated noise levels using a public chest X-ray data set. These findings are reproduced on an in-house noisy data set, where labels were extracted from the clinical systems of the University Hospital Bonn by a text-based transformer. The IDAC can therefore be a valuable tool for researchers, companies or clinics aiming to develop accurate and reliable DDSS from routine clinical data.