LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning

TL;DR

LLaMA-XR is a new framework combining LLaMA 3.1, DenseNet-121, and QLoRA fine-tuning to generate accurate, coherent, and efficient radiology reports from chest X-ray images, outperforming existing methods.

Contribution

The paper introduces LLaMA-XR, a novel, efficient framework that integrates advanced language models and image embeddings with QLoRA fine-tuning for improved radiology report generation.

Findings

LLaMA-XR achieves a ROUGE-L score of 0.433.

LLaMA-XR attains a METEOR score of 0.336.

Outperforms state-of-the-art methods on IU X-ray dataset.

Abstract

Automated radiology report generation holds significant potential to reduce radiologists' workload and enhance diagnostic accuracy. However, generating precise and clinically meaningful reports from chest radiographs remains challenging due to the complexity of medical language and the need for contextual understanding. Existing models often struggle with maintaining both accuracy and contextual relevance. In this paper, we present LLaMA-XR, a novel framework that integrates LLaMA 3.1 with DenseNet-121-based image embeddings and Quantized Low-Rank Adaptation (QLoRA) fine-tuning. LLaMA-XR achieves improved coherence and clinical accuracy while maintaining computational efficiency. This efficiency is driven by an optimization strategy that enhances parameter utilization and reduces memory overhead, enabling faster report generation with lower computational resource demands. Extensive experiments conducted on the IU X-ray benchmark dataset demonstrate that LLaMA-XR outperforms a range of state-of-the-art methods. Our model achieves a ROUGE-L score of 0.433 and a METEOR score of 0.336, establishing new performance benchmarks in the domain. These results underscore LLaMA-XR's potential as an effective and efficient AI system for automated radiology reporting, offering enhanced clinical utility and reliability.