Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules

TL;DR

This paper introduces CogVision, a dataset and interpretability framework to analyze attention heads in vision-language models, revealing their specialized roles in multimodal reasoning and their importance for model performance.

Contribution

It presents a novel methodology to identify and characterize functional attention heads in VLMs, linking them to reasoning processes and demonstrating their impact through intervention experiments.

Findings

Functional heads are sparse and vary across models.

Removing functional heads degrades multimodal reasoning performance.

Emphasizing functional heads improves accuracy.

Abstract

Despite excelling on multimodal benchmarks, vision-language models (VLMs) largely remain a black box. In this paper, we propose a novel interpretability framework to systematically analyze the internal mechanisms of VLMs, focusing on the functional roles of attention heads in multimodal reasoning. To this end, we introduce CogVision, a dataset that decomposes complex multimodal questions into step-by-step subquestions designed to simulate human reasoning through a chain-of-thought paradigm, with each subquestion associated with specific receptive or cognitive functions such as high-level visual reception and inference. Using a probing-based methodology, we identify attention heads that specialize in these functions and characterize them as functional heads. Our analysis across diverse VLM families reveals that these functional heads are universally sparse, vary in number and distribution across functions, and mediate interactions and hierarchical organization. Furthermore, intervention experiments demonstrate their critical role in multimodal reasoning: removing functional heads leads to performance degradation, while emphasizing them enhances accuracy. These findings provide new insights into the cognitive organization of VLMs and suggest promising directions for designing models with more human-aligned perceptual and reasoning abilities.