Learning to estimate label uncertainty for automatic radiology report parsing

TL;DR

This paper introduces a method to estimate label uncertainty in radiology report parsing, improving the training of image models by providing more nuanced labels than binary predictions.

Contribution

It proposes a Bidirectional LSTM model that augments heuristic report labels with uncertainty estimates, enhancing label quality for medical image analysis.

Findings

Achieves comparable or better performance than existing NLP tools

Provides uncertainty estimates for report labels

Improves downstream image model training efficiency

Abstract

Bootstrapping labels from radiology reports has become the scalable alternative to provide inexpensive ground truth for medical imaging. Because of the domain specific nature, state-of-the-art report labeling tools are predominantly rule-based. These tools, however, typically yield a binary 0 or 1 prediction that indicates the presence or absence of abnormalities. These hard targets are then used as ground truth to train image models in the downstream, forcing models to express high degree of certainty even on cases where specificity is low. This could negatively impact the statistical efficiency of image models. We address such an issue by training a Bidirectional Long-Short Term Memory Network to augment heuristic-based discrete labels of X-ray reports from all body regions and achieve performance comparable or better than domain-specific NLP, but with additional uncertainty estimates which enable finer downstream image model training.