MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generation

TL;DR

This paper introduces MCA-RG, a knowledge-driven framework that explicitly aligns visual features with medical concepts like pathology and anatomy to improve radiology report generation, achieving superior results on public benchmarks.

Contribution

MCA-RG is the first framework to explicitly align visual features with curated medical concept banks and incorporate contrastive learning and feature gating for enhanced report accuracy.

Findings

Outperforms existing methods on MIMIC-CXR and CheXpert Plus benchmarks.

Effectively aligns visual features with medical concepts for accurate report generation.

Improves generalization of anatomical features through contrastive learning.

Abstract

Despite significant advancements in adapting Large Language Models (LLMs) for radiology report generation (RRG), clinical adoption remains challenging due to difficulties in accurately mapping pathological and anatomical features to their corresponding text descriptions. Additionally, semantic agnostic feature extraction further hampers the generation of accurate diagnostic reports. To address these challenges, we introduce Medical Concept Aligned Radiology Report Generation (MCA-RG), a knowledge-driven framework that explicitly aligns visual features with distinct medical concepts to enhance the report generation process. MCA-RG utilizes two curated concept banks: a pathology bank containing lesion-related knowledge, and an anatomy bank with anatomical descriptions. The visual features are aligned with these medical concepts and undergo tailored enhancement. We further propose an anatomy-based contrastive learning procedure to improve the generalization of anatomical features, coupled with a matching loss for pathological features to prioritize clinically relevant regions. Additionally, a feature gating mechanism is employed to filter out low-quality concept features. Finally, the visual features are corresponding to individual medical concepts, and are leveraged to guide the report generation process. Experiments on two public benchmarks (MIMIC-CXR and CheXpert Plus) demonstrate that MCA-RG achieves superior performance, highlighting its effectiveness in radiology report generation.