Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learning

TL;DR

This paper introduces SCORE, a reinforcement learning-based adaptive token compression method that significantly accelerates video understanding tasks while maintaining high accuracy.

Contribution

SCORE is a novel reinforcement learning framework that learns dynamic token compression policies conditioned on temporal dynamics for efficient video understanding.

Findings

SCORE achieves a 16x prefill speedup on benchmarks.

SCORE retains 99.5% of original performance at 10% token retention.

SCORE outperforms existing compression methods in efficiency and accuracy.

Abstract

Multimodal Large Language Models have demonstrated remarkable capabilities in video understanding, yet face prohibitive computational costs and performance degradation from ''context rot'' due to massive visual token redundancy. Existing compression strategies typically rely on heuristics or fixed transformations that are often decoupled from the downstream task objectives, limiting their adaptability and effectiveness. To address this, we propose SCORE (Surprise-augmented token COmpression via REinforcement learning), a unified framework that learns an adaptive token compression policy. SCORE introduces a lightweight policy network conditioned on a surprise-augmented state representation that incorporates inter-frame residuals to explicitly capture temporal dynamics and motion saliency. We optimize this policy using a group-wise reinforcement learning scheme with a split-advantage estimator, stabilized by a two-stage curriculum transferring from static pseudo-videos to real dynamic videos. Extensive experiments on diverse video understanding benchmarks demonstrate that SCORE significantly outperforms state-of-the-art baselines. Notably, SCORE achieves a 16x prefill speedup while preserving 99.5% of original performance at a 10% retention ratio, offering a scalable solution for efficient long-form video understanding.