On the Risk of Misleading Reports: Diagnosing Textual Biases in Multimodal Clinical AI

TL;DR

This paper introduces a perturbation-based method called Selective Modality Shifting to diagnose and quantify biases in multimodal clinical AI models, revealing a tendency to over-rely on textual information over visual cues.

Contribution

The work presents a novel approach to systematically assess modality reliance in Vision-Language Models for medical data, highlighting the need for genuine multimodal integration.

Findings

Models predominantly depend on text over images.

Bias towards textual modality persists despite visual information.

Qualitative analysis confirms overshadowing of image content.

Abstract

Clinical decision-making relies on the integrated analysis of medical images and the associated clinical reports. While Vision-Language Models (VLMs) can offer a unified framework for such tasks, they can exhibit strong biases toward one modality, frequently overlooking critical visual cues in favor of textual information. In this work, we introduce Selective Modality Shifting (SMS), a perturbation-based approach to quantify a model's reliance on each modality in binary classification tasks. By systematically swapping images or text between samples with opposing labels, we expose modality-specific biases. We assess six open-source VLMs-four generalist models and two fine-tuned for medical data-on two medical imaging datasets with distinct modalities: MIMIC-CXR (chest X-ray) and FairVLMed (scanning laser ophthalmoscopy). By assessing model performance and the calibration of every model in both unperturbed and perturbed settings, we reveal a marked dependency on text input, which persists despite the presence of complementary visual information. We also perform a qualitative attention-based analysis which further confirms that image content is often overshadowed by text details. Our findings highlight the importance of designing and evaluating multimodal medical models that genuinely integrate visual and textual cues, rather than relying on single-modality signals.