Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages

TL;DR

This paper evaluates the performance of instruction-tuned Vision-Language Models in generating radiology reports across low-resource languages, emphasizing the importance of linguistic and domain-specific adaptations for improved accuracy.

Contribution

It introduces a comprehensive benchmark for assessing VLMs in low-resource radiology report generation and analyzes effective adaptation strategies for multilingual healthcare applications.

Findings

Language-specific models outperform general and domain-specific models.

Fine-tuning with medical terminology improves report quality.

Model temperature influences report coherence.

Abstract

The integration of artificial intelligence in healthcare has opened new horizons for improving medical diagnostics and patient care. However, challenges persist in developing systems capable of generating accurate and contextually relevant radiology reports, particularly in low-resource languages. In this study, we present a comprehensive benchmark to evaluate the performance of instruction-tuned Vision-Language Models (VLMs) in the specialized task of radiology report generation across three low-resource languages: Italian, German, and Spanish. Employing the LLaVA architectural framework, we conducted a systematic evaluation of pre-trained models utilizing general datasets, domain-specific datasets, and low-resource language-specific datasets. In light of the unavailability of models that possess prior knowledge of both the medical domain and low-resource languages, we analyzed various adaptations to determine the most effective approach for these contexts. The results revealed that language-specific models substantially outperformed both general and domain-specific models in generating radiology reports, emphasizing the critical role of linguistic adaptation. Additionally, models fine-tuned with medical terminology exhibited enhanced performance across all languages compared to models with generic knowledge, highlighting the importance of domain-specific training. We also explored the influence of the temperature parameter on the coherence of report generation, providing insights for optimal model settings. Our findings highlight the importance of tailored language and domain-specific training for improving the quality and accuracy of radiological reports in multilingual settings. This research not only advances our understanding of VLMs adaptability in healthcare but also points to significant avenues for future investigations into model tuning and language-specific adaptations.