VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition

TL;DR

VideoCuRL introduces a novel curriculum learning framework for video reinforcement learning that decomposes difficulty into visual perception and cognitive reasoning axes, improving training stability and performance.

Contribution

It proposes a two-dimensional difficulty decomposition and a competence-aware scheduling strategy, along with stabilization techniques, to enhance video RL training efficiency and effectiveness.

Findings

Outperforms strong RL baselines on reasoning and perception tasks.

Eliminates inference overhead of generation-based curricula.

Improves training stability with Dynamic Sparse KL and Structured Revisiting.

Abstract

Reinforcement Learning (RL) is crucial for empowering VideoLLMs with complex spatiotemporal reasoning. However, current RL paradigms predominantly rely on random data shuffling or naive curriculum strategies based on scalar difficulty metrics. We argue that scalar metrics fail to disentangle two orthogonal challenges in video understanding: Visual Temporal Perception Load and Cognitive Reasoning Depth. To address this, we propose VideoCuRL, a novel framework that decomposes difficulty into these two axes. We employ efficient, training-free proxies, optical flow and keyframe entropy for visual complexity, Calibrated Surprisal for cognitive complexity, to map data onto a 2D curriculum grid. A competence aware Diagonal Wavefront strategy then schedules training from base alignment to complex reasoning. Furthermore, we introduce Dynamic Sparse KL and Structured Revisiting to stabilize training against reward collapse and catastrophic forgetting. Extensive experiments show that VideoCuRL surpasses strong RL baselines on reasoning (+2.5 on VSI-Bench) and perception (+2.9 on VideoMME) tasks. Notably, VideoCuRL eliminates the prohibitive inference overhead of generation-based curricula, offering a scalable solution for robust video post-training.