EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge

TL;DR

EVOKE is a novel framework for chest X-ray report generation that leverages multi-view contrastive learning and patient-specific knowledge to improve accuracy and coherence, surpassing existing methods.

Contribution

The paper introduces EVOKE, integrating multi-view contrastive learning and patient-specific knowledge into radiology report generation, along with new datasets for multi-view X-ray analysis.

Findings

Achieved 2.9% F1 RadGraph improvement on MIMIC-CXR

Achieved 7.3% BLEU-1 improvement on MIMIC-ABN

Achieved 8.2% F1 CheXbert improvement on Two-view CXR

Abstract

Radiology reports are crucial for planning treatment strategies and facilitating effective doctor-patient communication. However, the manual creation of these reports places a significant burden on radiologists. While automatic radiology report generation presents a promising solution, existing methods often rely on single-view radiographs, which constrain diagnostic accuracy. To address this challenge, we propose \textbf{EVOKE}, a novel chest X-ray report generation framework that incorporates multi-view contrastive learning and patient-specific knowledge. Specifically, we introduce a multi-view contrastive learning method that enhances visual representation by aligning multi-view radiographs with their corresponding report. After that, we present a knowledge-guided report generation module that integrates available patient-specific indications (e.g., symptom descriptions) to trigger the production of accurate and coherent radiology reports. To support research in multi-view report generation, we construct Multi-view CXR and Two-view CXR datasets using publicly available sources. Our proposed EVOKE surpasses recent state-of-the-art methods across multiple datasets, achieving a 2.9\% F\textsubscript{1} RadGraph improvement on MIMIC-CXR, a 7.3\% BLEU-1 improvement on MIMIC-ABN, a 3.1\% BLEU-4 improvement on Multi-view CXR, and an 8.2\% F\textsubscript{1,mic-14} CheXbert improvement on Two-view CXR.