CircuitProbe: Tracing Visual Temporal Evidence Flow in Video Language Models

TL;DR

CircuitProbe is a novel analysis framework that dissects how video-language models represent and utilize temporal information, leading to targeted interventions that improve temporal understanding in these models.

Contribution

We introduce CircuitProbe, a circuit-level analysis method that localizes and traces temporal evidence in video-language models, enabling effective interventions for temporal reasoning.

Findings

Identifies temporally specialized attention heads in LVLMs.

Targeted interventions improve temporal understanding by up to 2.4%.

Validates the analysis framework's effectiveness on TempCompass benchmark.

Abstract

Autoregressive large vision--language models (LVLMs) interface video and language by projecting video features into the LLM's embedding space as continuous visual token embeddings. However, it remains unclear where temporal evidence is represented and how it causally influences decoding. To address this gap, we present CircuitProbe, a circuit-level analysis framework that dissects the end-to-end video-language pathway through two stages: (i) Visual Auditing, which localizes object semantics within the projected video-token sequence and reveals their causal necessity via targeted ablations and controlled substitutions; and (ii) Semantic Tracing, which uses logit-lens probing to track the layer-wise emergence of object and temporal concepts, augmented with temporal frame interventions to assess sensitivity to temporal structure. Based on the resulting analysis, we design a targeted surgical intervention that strictly follows our observations: identifying temporally specialized attention heads and selectively amplifying them within the critical layer interval revealed by Semantic Tracing. This analysis-driven intervention yields consistent improvements (up to 2.4% absolute) on the temporal-heavy TempCompass benchmark, validating the correctness, effectiveness, and practical value of the proposed circuit-level analysis for temporal understanding in LVLMs.