ViX-Ray: A Vietnamese Chest X-Ray Dataset for Vision-Language Models

TL;DR

ViX-Ray introduces a Vietnamese chest X-ray dataset with expert annotations, highlighting linguistic features and evaluating vision-language models' performance, revealing challenges like hallucination and low precision in clinical report generation.

Contribution

The paper presents ViX-Ray, the first Vietnamese chest X-ray dataset with annotations, and benchmarks multiple models, revealing linguistic insights and challenges in Vietnamese medical VLMs.

Findings

Models often hallucinate and lack precision in report generation.

ViX-Ray reveals unique linguistic patterns in Vietnamese radiology reports.

Benchmark results highlight the need for improved Vietnamese medical VLMs.

Abstract

Vietnamese medical research has become an increasingly vital domain, particularly with the rise of intelligent technologies aimed at reducing time and resource burdens in clinical diagnosis. Recent advances in vision-language models (VLMs), such as Gemini and GPT-4V, have sparked a growing interest in applying AI to healthcare. However, most existing VLMs lack exposure to Vietnamese medical data, limiting their ability to generate accurate and contextually appropriate diagnostic outputs for Vietnamese patients. To address this challenge, we introduce ViX-Ray, a novel dataset comprising 5,400 Vietnamese chest X-ray images annotated with expert-written findings and impressions from physicians at a major Vietnamese hospital. We analyze linguistic patterns within the dataset, including the frequency of mentioned body parts and diagnoses, to identify domain-specific linguistic characteristics of Vietnamese radiology reports. Furthermore, we fine-tune five state-of-the-art open-source VLMs on ViX-Ray and compare their performance to leading proprietary models, GPT-4V and Gemini. Our results show that while several models generate outputs partially aligned with clinical ground truths, they often suffer from low precision and excessive hallucination, especially in impression generation. These findings not only demonstrate the complexity and challenge of our dataset but also establish ViX-Ray as a valuable benchmark for evaluating and advancing vision-language models in the Vietnamese clinical domain.