ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

TL;DR

ReFoCUS introduces a reinforcement learning framework for optimizing frame selection in video understanding, improving reasoning performance by aligning frame choices with model-internal preferences without explicit supervision.

Contribution

It proposes a novel reinforcement-guided frame optimization method that enhances video reasoning by learning a policy for selecting relevant frames based on model feedback.

Findings

Improves reasoning performance across multiple video QA benchmarks.

Learns a frame selection policy without explicit supervision.

Aligns frame selection with model-internal utility signals.

Abstract

Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to understand video content remains constrained by suboptimal frame selection strategies. Existing approaches often rely on static heuristics or external retrieval modules to feed frame information into video-LLMs, which may fail to provide the query-relevant information. In this work, we introduce ReFoCUS (Reinforcement-guided Frame Optimization for Contextual UnderStanding), a novel frame-level policy optimization framework that shifts the optimization target from textual responses to visual input selection. ReFoCUS learns a frame selection policy via reinforcement learning, using reward signals derived from a reference LMM to reflect the model's intrinsic preferences for frames that best support temporally grounded responses. To efficiently explore the large combinatorial frame space, we employ an autoregressive, conditional selection architecture that ensures temporal coherence while reducing complexity. Our approach does not require explicit supervision at the frame-level and consistently improves reasoning performance across multiple video QA benchmarks, highlighting the benefits of aligning frame selection with model-internal utility.