ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding

TL;DR

ReAgent-V is a flexible, reward-driven multi-agent framework that improves video understanding by enabling real-time feedback, iterative reasoning, and data filtering, leading to significant performance gains across multiple tasks.

Contribution

The paper introduces ReAgent-V, a novel framework integrating real-time reward signals and multi-perspective reflection for enhanced video understanding and reasoning.

Findings

Achieves up to 6.9% improvement in video understanding accuracy.

Enhances reasoning capabilities across 12 datasets.

Supports flexible tool integration for diverse tasks.

Abstract

Video understanding is fundamental to tasks such as action recognition, video reasoning, and robotic control. Early video understanding methods based on large vision-language models (LVLMs) typically adopt a single-pass reasoning paradigm without dynamic feedback, limiting the model's capacity to self-correct and adapt in complex scenarios. Recent efforts have attempted to address this limitation by incorporating reward models and reinforcement learning to enhance reasoning, or by employing tool-agent frameworks. However, these approaches face several challenges, including high annotation costs, reward signals that fail to capture real-time reasoning states, and low inference efficiency. To overcome these issues, we propose ReAgent-V, a novel agentic video understanding framework that integrates efficient frame selection with real-time reward generation during inference. These reward signals not only guide iterative answer refinement through a multi-perspective reflection mechanism-adjusting predictions from conservative, neutral, and aggressive viewpoints-but also enable automatic filtering of high-quality data for supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). ReAgent-V is lightweight, modular, and extensible, supporting flexible tool integration tailored to diverse tasks. Extensive experiments on 12 datasets across three core applications-video understanding, video reasoning enhancement, and vision-language-action model alignment-demonstrate significant gains in generalization and reasoning, with improvements of up to 6.9%, 2.1%, and 9.8%, respectively, highlighting the effectiveness and versatility of the proposed framework.