Effect of Radiology Report Labeler Quality on Deep Learning Models for Chest X-Ray Interpretation

TL;DR

This study evaluates how the quality of radiology report labelers affects deep learning model performance in chest X-ray interpretation, showing that better labelers lead to more accurate classification models.

Contribution

It systematically compares different radiology report labelers and demonstrates that higher-quality labels improve deep learning model accuracy for chest X-ray classification.

Findings

VisualCheXbert outperforms CheXpert and CheXbert labelers in label accuracy.

Models trained on VisualCheXbert labels achieve higher classification performance.

Improved report labeling methods enhance deep learning model effectiveness.

Abstract

Although deep learning models for chest X-ray interpretation are commonly trained on labels generated by automatic radiology report labelers, the impact of improvements in report labeling on the performance of chest X-ray classification models has not been systematically investigated. We first compare the CheXpert, CheXbert, and VisualCheXbert labelers on the task of extracting accurate chest X-ray image labels from radiology reports, reporting that the VisualCheXbert labeler outperforms the CheXpert and CheXbert labelers. Next, after training image classification models using labels generated from the different radiology report labelers on one of the largest datasets of chest X-rays, we show that an image classification model trained on labels from the VisualCheXbert labeler outperforms image classification models trained on labels from the CheXpert and CheXbert labelers. Our work suggests that recent improvements in radiology report labeling can translate to the development of higher performing chest X-ray classification models.