KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models

TL;DR

KARGEN enhances automated radiology report generation by integrating a knowledge graph with large language models, improving disease sensitivity and report quality in clinical settings.

Contribution

This paper introduces KARGEN, a novel framework that combines knowledge graphs with LLMs to produce more accurate and disease-aware radiology reports.

Findings

Improved report quality on MIMIC-CXR and IU-Xray datasets.

Enhanced disease sensitivity in generated reports.

Effective feature fusion methods for clinical relevance.

Abstract

Harnessing the robust capabilities of Large Language Models (LLMs) for narrative generation, logical reasoning, and common-sense knowledge integration, this study delves into utilizing LLMs to enhance automated radiology report generation (R2Gen). Despite the wealth of knowledge within LLMs, efficiently triggering relevant knowledge within these large models for specific tasks like R2Gen poses a critical research challenge. This paper presents KARGEN, a Knowledge-enhanced Automated radiology Report GENeration framework based on LLMs. Utilizing a frozen LLM to generate reports, the framework integrates a knowledge graph to unlock chest disease-related knowledge within the LLM to enhance the clinical utility of generated reports. This is achieved by leveraging the knowledge graph to distill disease-related features in a designed way. Since a radiology report encompasses both normal and disease-related findings, the extracted graph-enhanced disease-related features are integrated with regional image features, attending to both aspects. We explore two fusion methods to automatically prioritize and select the most relevant features. The fused features are employed by LLM to generate reports that are more sensitive to diseases and of improved quality. Our approach demonstrates promising results on the MIMIC-CXR and IU-Xray datasets.