Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models

TL;DR

This paper investigates the performance gap in vision-language models when extracting entity knowledge from images versus text, revealing significant accuracy drops and analyzing internal information flow to suggest improvements.

Contribution

The study introduces the PopVQA dataset to benchmark entity knowledge extraction and analyzes internal model mechanics to identify bottlenecks in information flow from images to reasoning.

Findings

Accuracy drops up to 18% for visual entity questions

Meaningful information flow occurs mainly in deeper layers

Critical image processing happens in the middle layers of the language model

Abstract

Vision-language models (VLMs) excel at extracting and reasoning about information from images. Yet, their capacity to leverage internal knowledge about specific entities remains underexplored. This work investigates the disparity in model performance when answering factual questions about an entity described in text versus depicted in an image. Our results reveal a significant accuracy drop - reaching 18% for some models - when the entity is presented visually instead of textually. To study this gap we present PopVQA, a dataset which allows separating entity recognition and question answering, and use it to benchmark several models. We hypothesize that this decline arises from limitations in how information flows from image tokens to query tokens. Thus, we use mechanistic interpretability tools to reveal that, although image tokens are preprocessed by the vision encoder, meaningful information flow from these tokens occurs only in the much deeper layers. Furthermore, critical image processing happens in the language model's middle layers, allowing few layers for consecutive reasoning, highlighting a potential inefficiency in how the model utilizes its layers for reasoning. These insights shed light on the internal mechanics of VLMs and offer pathways for enhancing their reasoning capabilities. PopVQA can be found at https://huggingface.co/datasets/idoco/PopVQA.