The Geometry of Representational Failures in Vision Language Models

TL;DR

This paper investigates the internal geometric structure of vision-language models to understand their failures in multi-object visual tasks, revealing how concept representations influence errors.

Contribution

It introduces a geometric analysis of concept vectors in VLMs and demonstrates how their overlap correlates with specific visual errors, providing mechanistic insights.

Findings

Concept vectors can be manipulated to steer model behavior.

Overlap of concept vectors correlates with error patterns.

Geometric analysis explains visual failures in VLMs.

Abstract

Vision-Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing to identify the most similar objects among distractions. While these errors mirror human cognitive constraints, such as the "Binding Problem", the internal mechanisms driving them in artificial systems remain poorly understood. Here, we propose a mechanistic insight by analyzing the representational geometry of open-weight VLMs (Qwen, InternVL, Gemma), comparing methodologies to distill "concept vectors" - latent directions encoding visual concepts. We validate our concept vectors via steering interventions that reliably manipulate model behavior in both simplified and naturalistic vision tasks (e.g., forcing the model to perceive a red flower as blue). We observe that the geometric overlap between these vectors strongly correlates with specific error patterns, offering a grounded quantitative framework to understand how internal representations shape model behavior and drive visual failures.