Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation

TL;DR

This paper introduces SEI, a novel method for chest X-ray report generation that improves factual accuracy and clinical relevance by extracting structural entities and incorporating patient indications, validated on MIMIC-CXR.

Contribution

The paper presents a new structural entities extraction approach and a cross-modal fusion network that enhance report quality and clinical applicability in X-ray report generation.

Findings

SEI outperforms state-of-the-art methods on natural language metrics.

SEI improves clinical efficacy in report generation.

The method effectively incorporates patient indications.

Abstract

The automated generation of imaging reports proves invaluable in alleviating the workload of radiologists. A clinically applicable reports generation algorithm should demonstrate its effectiveness in producing reports that accurately describe radiology findings and attend to patient-specific indications. In this paper, we introduce a novel method, \textbf{S}tructural \textbf{E}ntities extraction and patient indications \textbf{I}ncorporation (SEI) for chest X-ray report generation. Specifically, we employ a structural entities extraction (SEE) approach to eliminate presentation-style vocabulary in reports and improve the quality of factual entity sequences. This reduces the noise in the following cross-modal alignment module by aligning X-ray images with factual entity sequences in reports, thereby enhancing the precision of cross-modal alignment and further aiding the model in gradient-free retrieval of similar historical cases. Subsequently, we propose a cross-modal fusion network to integrate information from X-ray images, similar historical cases, and patient-specific indications. This process allows the text decoder to attend to discriminative features of X-ray images, assimilate historical diagnostic information from similar cases, and understand the examination intention of patients. This, in turn, assists in triggering the text decoder to produce high-quality reports. Experiments conducted on MIMIC-CXR validate the superiority of SEI over state-of-the-art approaches on both natural language generation and clinical efficacy metrics.