Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning

TL;DR

This paper introduces VITAL, a novel multimodal reasoning framework that enhances long video understanding by densely sampling frames, generating multimodal chain-of-thoughts, and employing multi-task reinforcement learning to outperform existing methods.

Contribution

The paper presents VITAL, an end-to-end agentic video reasoning model with a visual toolbox, new datasets, and a difficulty-aware reinforcement learning algorithm, advancing long video reasoning capabilities.

Findings

VITAL outperforms existing methods on 11 video understanding benchmarks.

The model effectively handles long videos and complex reasoning chains.

Multi-task training improves performance in question answering and temporal grounding.

Abstract

The video reasoning ability of multimodal large language models (MLLMs) is crucial for downstream tasks like video question answering and temporal grounding. While recent approaches have explored text-based chain-of-thought (CoT) reasoning for MLLMs, these methods often suffer from limited cross-modal interaction and increased hallucination, especially with longer videos or reasoning chains. To address these challenges, we propose Video Intelligence via Tool-Augmented Learning (VITAL), a novel end-to-end agentic video reasoning framework. With a visual toolbox, the model can densely sample new video frames on demand and generate multimodal CoT for precise long video reasoning. We observe that temporal grounding and question answering are mutually beneficial for video understanding tasks. Therefore, we construct two high-quality multi-task video reasoning datasets MTVR-CoT-72k for supervised fine-tuning and MTVR-RL-110k for reinforcement learning. Moreover, we propose a Difficulty-aware Group Relative Policy Optimization algorithm (DGRPO) to mitigate difficulty imbalance in multi-task reinforcement learning. Extensive experiments on 11 challenging video understanding benchmarks demonstrate the advanced reasoning ability of VITAL, outperforming existing methods in video question answering and temporal grounding tasks, especially in long video scenarios. Code is available at https://zhang9302002.github.io/thinkingwithvideos-page/.