Video-KTR: Reinforcing Video Reasoning via Key Token Attribution

TL;DR

Video-KTR introduces a fine-grained, token-level reinforcement learning framework that enhances video reasoning accuracy and interpretability by focusing on key visual, temporal, and uncertain tokens, achieving state-of-the-art results.

Contribution

It presents a novel modality-aware policy shaping method that selectively reinforces informative tokens based on multiple attribution signals for improved video reasoning.

Findings

Achieves 42.7% on Video-Holmes benchmark, surpassing GPT-4o.

Demonstrates consistent improvements across multiple reasoning and understanding tasks.

Ablation studies confirm the effectiveness of combined attribution signals.

Abstract

Reinforcement learning (RL) has shown strong potential for enhancing reasoning in multimodal large language models, yet existing video reasoning methods often rely on coarse sequence-level rewards or single-factor token selection, neglecting fine-grained links among visual inputs, temporal dynamics, and linguistic outputs, limiting both accuracy and interpretability. We propose Video-KTR, a modality-aware policy shaping framework that performs selective, token-level RL by combining three attribution signals: (1) visual-aware tokens identified via counterfactual masking to reveal perceptual dependence; (2) temporal-aware tokens detected through frame shuffling to expose temporal sensitivity; and (3) high-entropy tokens signaling predictive uncertainty. By reinforcing only these key tokens, Video-KTR focuses learning on semantically informative, modality-sensitive content while filtering out low-value tokens. Across five challenging benchmarks, Video-KTR achieves state-of-the-art or highly competitive results, achieving 42.7\% on Video-Holmes (surpassing GPT-4o) with consistent gains on both reasoning and general video understanding tasks. Ablation studies verify the complementary roles of the attribution signals and the robustness of targeted token-level updates. Overall, Video-KTR improves accuracy and interpretability, offering a simple, drop-in extension to RL for complex video reasoning. Our code and models are available at https://github.com/zywang0104/Video-KTR.