Understanding Counting Mechanisms in Large Language and Vision-Language Models

TL;DR

This paper investigates how large language and vision-language models represent and process counting, revealing a layered, structured internal mechanism that relies on tokens, visual features, and structural cues.

Contribution

It introduces CountScope, a tool for interpretability, and uncovers the layerwise emergence of numerical representations and internal counters in LLMs and LVLMs.

Findings

Tokens and visual features encode count information.

Numerical representations emerge progressively across layers.

Models use structural cues like separators as counting shortcuts.

Abstract

Counting is one of the fundamental abilities of large language models (LLMs) and large vision-language models (LVLMs). This paper examines how these foundation models represent and compute numerical information in counting tasks. We use controlled experiments with repeated textual and visual items and analyze counting in LLMs and LVLMs through a set of behavioral, observational, and causal mediation analyses. To this end, we design a specialized tool, CountScope, for the mechanistic interpretability of numerical content. Results show that individual tokens or visual features encode latent positional count information that can be extracted and transferred across contexts. Layerwise analyses reveal a progressive emergence of numerical representations, with lower layers encoding small counts and higher layers representing larger ones. We identify an internal counter mechanism that updates with each item, stored mainly in the final token or region. In LVLMs, numerical information also appears in visual embeddings, shifting between background and foreground regions depending on spatial composition. We further reveal that models rely on structural cues such as separators in text, which act as shortcuts for tracking item counts and strongly influence the accuracy of numerical predictions. Overall, counting emerges as a structured, layerwise process in LLMs and follows the same general pattern in LVLMs, shaped by the properties of the vision encoder.