Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs

TL;DR

This paper investigates why vision-language models sometimes fail despite perceiving the correct visual evidence, revealing that they often see but do not believe the evidence, and proposes an attention-based intervention to improve accuracy.

Contribution

It uncovers the disconnect between evidence perception and utilization in VLMs and introduces a training-free method to enhance their reasoning by highlighting evidence regions.

Findings

Deeper layers reliably attend to evidence regions

VLMs often perceive evidence even when answers are incorrect

Selective attention masking improves model accuracy

Abstract

Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing'' that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.