EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs

TL;DR

EchoPrune is a lightweight, training-free token pruning method that interprets redundant video tokens as temporal echoes, enabling more efficient and accurate long-form video understanding with VideoLLMs.

Contribution

It introduces a novel approach to token pruning by leveraging the concept of temporal echoes, improving temporal resolution without additional computational overhead.

Findings

Enables processing up to 20x more frames under the same token budget.

Improves performance by +8.6% on multiple video understanding benchmarks.

Achieves 5.6x inference speedup for prefilled decoding.

Abstract

Long-form video understanding remains challenging for Video Large Language Models (VideoLLMs), as the dense frame sampling introduces massive visual tokens while sparse sampling risks missing critical temporal evidence and leading to LLM hallucination. Existing training-free token reduction methods either treat videos equally as static images or rely on segment-level merging heuristics, which weaken fine-grained spatiotemporal modeling and introduce additional overhead. In this paper, we propose EchoPrune, a lightweight and training-free token pruning method that improves temporal resolution under a fixed LLM-side visual token budget. Our core idea is to interpret redundant video tokens as temporal echoes: if a token is well reconstructed from the previous frame, it is merely a temporally redundant echo; otherwise, it may capture new events, motion, or query-relevant visual evidence. Based on this insight, EchoPrune scores visual tokens by (i) query-guided crossmodal relevance and (ii) temporal reconstruction error, measured by correspondence matching and echo matching across consecutive frames. The selected tokens preserve task-relevant cues and temporal novelty while suppressing predictable redundancy, allowing VideoLLMs to observe more frames without increasing the decoding budget. Extensive experiments on LLaVA-OV, Qwen2.5VL, and Qwen3VL across six video understanding benchmarks show that EchoPrune enables VideoLLMs to process up to 20x frames under the same token budget, yielding improved performance (+8.6%) and inference speedup (5.6x for prefilling) on Qwen2.5VL-7B.