Expert-aware uncertainty estimation for quality control of neural-based blood typing

TL;DR

This paper introduces a novel expert-aware method for uncertainty estimation in neural blood typing models, improving prediction accuracy by incorporating medical expert assessments of case difficulty.

Contribution

It presents a new approach that integrates expert complexity ratings into neural training, addressing the challenge of unlabeled case hardness in medical diagnostics.

Findings

2.5-fold improvement in uncertainty prediction with expert labels

35% increase in performance using neural-based expert consensus

Validated on a new dataset 'BloodyWell' with expert annotations

Abstract

In medical diagnostics, accurate uncertainty estimation for neural-based models is essential for complementing second-opinion systems. Despite neural network ensembles' proficiency in this problem, a gap persists between actual uncertainties and predicted estimates. A major difficulty here is the lack of labels on the hardness of examples: a typical dataset includes only ground truth target labels, making the uncertainty estimation problem almost unsupervised. Our novel approach narrows this gap by integrating expert assessments of case complexity into the neural network's learning process, utilizing both definitive target labels and supplementary complexity ratings. We validate our methodology for blood typing, leveraging a new dataset "BloodyWell" unique in augmenting labeled reaction images with complexity scores from six medical specialists. Experiments demonstrate enhancement of our approach in uncertainty prediction, achieving a 2.5-fold improvement with expert labels and a 35% increase in performance with estimates of neural-based expert consensus.