MATEX: Multi-scale Attention and Text-guided Explainability of Medical Vision-Language Models

TL;DR

MATEX is a novel framework that improves interpretability of medical vision-language models by integrating multi-scale attention, text-guided spatial priors, and layer analysis to produce accurate and clinically meaningful explanations.

Contribution

It introduces a new interpretability method that combines attention mechanisms and anatomical priors, addressing spatial imprecision and lack of grounding in prior approaches.

Findings

MATEX outperforms M2IB in spatial precision.

MATEX produces more stable and clinically aligned explanations.

Enhanced trust in radiological AI applications.

Abstract

We introduce MATEX (Multi-scale Attention and Text-guided Explainability), a novel framework that advances interpretability in medical vision-language models by incorporating anatomically informed spatial reasoning. MATEX synergistically combines multi-layer attention rollout, text-guided spatial priors, and layer consistency analysis to produce precise, stable, and clinically meaningful gradient attribution maps. By addressing key limitations of prior methods, such as spatial imprecision, lack of anatomical grounding, and limited attention granularity, MATEX enables more faithful and interpretable model explanations. Evaluated on the MS-CXR dataset, MATEX outperforms the state-of-the-art M2IB approach in both spatial precision and alignment with expert-annotated findings. These results highlight MATEX's potential to enhance trust and transparency in radiological AI applications.