APPO: Attention-guided Perception Policy Optimization for Video Reasoning

TL;DR

This paper introduces APPO, a novel method that enhances video reasoning models' perception abilities by optimizing token-level perception through intra-group perception tokens, leading to improved performance across benchmarks.

Contribution

We propose APPO, a perception policy optimization algorithm that leverages token-level dense rewards to improve perception without expensive annotations, outperforming existing methods.

Findings

APPO outperforms GRPO and DAPO by 0.5% to 4% on various benchmarks.

Enhancing perception ability yields greater performance gains than increasing reasoning complexity.

APPO effectively improves perception in models with different scales (3/7B).

Abstract

Complex video reasoning, actually, relies excessively on fine-grained perception rather than on expert (e.g., Ph.D, Science)-level reasoning. Through extensive empirical observation, we have recognized the critical impact of perception. In particular, when perception ability is almost fixed, enhancing reasoning from Qwen3-8B to OpenAI-o3 yields only 0.7% performance improvement. Conversely, even minimal change in perception model scale (from 7B to 32B) boosts performance by 1.4%, indicating enhancing perception, rather than reasoning, is more critical to improve performance. Therefore, exploring how to enhance perception ability through reasoning without the need for expensive fine-grained annotation information is worthwhile. To achieve this goal, we specially propose APPO, the Attention-guided Perception Policy Optimization algorithm that leverages token-level dense rewards to improve model's fine-grained perception. The core idea behind APPO is to optimize those tokens from different responses that primarily focus on the same crucial video frame (called intra-group perception tokens). Experimental results on diverse video benchmarks and models with different scales (3/7B) demonstrate APPO consistently outperforms GRPO and DAPO (0.5%~4%). We hope our work provides a promising approach to effectively enhance model's perception abilities through reasoning in a low-cost manner, serving diverse scenarios and demands.