LLM-RG4: Flexible and Factual Radiology Report Generation across Diverse Input Contexts

TL;DR

This paper introduces LLM-RG4, a flexible and factual radiology report generation framework that adapts to diverse input scenarios, reduces hallucinations, and achieves state-of-the-art results on clinical datasets.

Contribution

The paper develops a new dataset and proposes a novel LLM-based framework with adaptive token fusion and loss weighting for flexible, accurate radiology report generation.

Findings

Achieves state-of-the-art performance on MIMIC-RG4 and MIMIC-CXR datasets.

Reduces input-agnostic hallucinations compared to existing models.

Handles diverse input scenarios effectively with minimal computational overhead.

Abstract

Drafting radiology reports is a complex task requiring flexibility, where radiologists tail content to available information and particular clinical demands. However, most current radiology report generation (RRG) models are constrained to a fixed task paradigm, such as predicting the full ``finding'' section from a single image, inherently involving a mismatch between inputs and outputs. The trained models lack the flexibility for diverse inputs and could generate harmful, input-agnostic hallucinations. To bridge the gap between current RRG models and the clinical demands in practice, we first develop a data generation pipeline to create a new MIMIC-RG4 dataset, which considers four common radiology report drafting scenarios and has perfectly corresponded input and output. Secondly, we propose a novel large language model (LLM) based RRG framework, namely LLM-RG4, which utilizes LLM's flexible instruction-following capabilities and extensive general knowledge. We further develop an adaptive token fusion module that offers flexibility to handle diverse scenarios with different input combinations, while minimizing the additional computational burden associated with increased input volumes. Besides, we propose a token-level loss weighting strategy to direct the model's attention towards positive and uncertain descriptions. Experimental results demonstrate that LLM-RG4 achieves state-of-the-art performance in both clinical efficiency and natural language generation on the MIMIC-RG4 and MIMIC-CXR datasets. We quantitatively demonstrate that our model has minimal input-agnostic hallucinations, whereas current open-source models commonly suffer from this problem.