Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models

TL;DR

This paper evaluates the faithfulness of visual attribution methods in chest X-ray reasoning by developing a causal framework and proposing MedFocus, a new concept-based attribution method that improves trustworthiness.

Contribution

It introduces a causal evaluation framework for visual attribution in medical LVLMs and proposes MedFocus, a novel method that enhances explanation accuracy and clinical relevance.

Findings

Existing attribution methods often fail to identify the true evidence used by LVLMs.

MedFocus significantly outperforms prior attribution methods in localizing clinically meaningful regions.

The framework and MedFocus improve the trustworthiness of model explanations in medical imaging.

Abstract

Large Vision Language Models (LVLMs) show promise in medical applications, but their inability to faithfully ground responses in visual evidence raises serious concerns about clinical trustworthiness. While visual attribution methods are widely used to explain LVLM predictions, whether these explanations actually reflect the visual evidence underlying the model's decision is largely unverified, since ground-truth annotations for internal model reasoning are typically unavailable. We address this question for chest X-ray (CXR) reasoning by developing a causal evaluation framework that retains only CXR-VQA samples for which the expert-annotated region is verified, via counterfactual editing, to be causally responsible for the model's prediction. Using this framework across 11 attribution methods, six open-source LVLMs, and two output modes (direct answer and step-by-step reasoning), we find that existing attribution methods often fail to identify the evidence used by LVLMs. To address this failure, we propose MedFocus, a concept-based attribution method that localizes clinically meaningful anatomical regions via unbalanced optimal transport and measures their causal effect on model outputs through targeted interventions. MedFocus produces spatial, concept-level, and token-level attributions and substantially outperforms prior methods, taking a step toward more trustworthy attribution for medical LVLMs. Our data and code are available at https://github.com/gzxiong/medfocus/.