ReXErr: Synthesizing Clinically Meaningful Errors in Diagnostic Radiology Reports

TL;DR

ReXErr is a novel method using Large Language Models to generate realistic, diverse errors in radiology reports, aiding in improving report accuracy and evaluation of correction algorithms.

Contribution

The paper introduces ReXErr, a new approach for synthesizing clinically meaningful errors in radiology reports using LLMs, with a focus on error diversity and clinical plausibility.

Findings

ReXErr produces errors that closely mimic real-world mistakes.

The method is consistent across different error categories.

ReXErr can be used to evaluate report correction algorithms.

Abstract

Accurately interpreting medical images and writing radiology reports is a critical but challenging task in healthcare. Both human-written and AI-generated reports can contain errors, ranging from clinical inaccuracies to linguistic mistakes. To address this, we introduce ReXErr, a methodology that leverages Large Language Models to generate representative errors within chest X-ray reports. Working with board-certified radiologists, we developed error categories that capture common mistakes in both human and AI-generated reports. Our approach uses a novel sampling scheme to inject diverse errors while maintaining clinical plausibility. ReXErr demonstrates consistency across error categories and produces errors that closely mimic those found in real-world scenarios. This method has the potential to aid in the development and evaluation of report correction algorithms, potentially enhancing the quality and reliability of radiology reporting.