Semantically Informed Salient Regions Guided Radiology Report Generation

TL;DR

This paper introduces SISRNet, a novel method that uses semantically informed salient regions to improve the accuracy of radiology report generation from chest X-rays, effectively addressing data bias and subtle abnormal findings.

Contribution

SISRNet explicitly identifies and focuses on medically critical salient regions, enhancing report accuracy and robustness against data bias in radiology image analysis.

Findings

Outperforms existing methods on IU-Xray and MIMIC-CXR datasets.

Effectively captures subtle abnormal findings in chest X-rays.

Reduces the impact of data bias in report generation.

Abstract

Recent advances in automated radiology report generation from chest X-rays using deep learning algorithms have the potential to significantly reduce the arduous workload of radiologists. However, due to the inherent massive data bias in radiology images, where abnormalities are typically subtle and sparsely distributed, existing methods often produce fluent yet medically inaccurate reports, limiting their applicability in clinical practice. To address this issue effectively, we propose a Semantically Informed Salient Regions-guided (SISRNet) report generation method. Specifically, our approach explicitly identifies salient regions with medically critical characteristics using fine-grained cross-modal semantics. Then, SISRNet systematically focuses on these high-information regions during both image modeling and report generation, effectively capturing subtle abnormal findings, mitigating the negative impact of data bias, and ultimately generating clinically accurate reports. Compared to its peers, SISRNet demonstrates superior performance on widely used IU-Xray and MIMIC-CXR datasets.