VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

TL;DR

VideoTIR introduces a reinforcement learning-based approach to improve long video understanding by efficiently managing tool usage, reducing hallucinations, and enhancing accuracy and efficiency in multimodal large language models.

Contribution

The paper proposes a novel RL framework with TAGPO and a trajectory synthesis method to optimize tool usage for long video understanding in MLLMs.

Findings

Outperforms existing methods on three long-video QA benchmarks.

Reduces redundant tool-calling and improves efficiency.

Enhances understanding accuracy in long videos.

Abstract

Existing Multimodal Large Language Models (MLLMs) often suffer from hallucinations in long video understanding (LVU), primarily due to the imbalance between textual and visual tokens. Observing that MLLMs handle short visual inputs well, recent LVU works alleviate hallucinations by automatically parsing the vast visual data into manageable segments that can be effectively processed by MLLMs. SFT-based tool-calling methods can serve this purpose, but they typically require vast amounts of fine-grained, high-quality data and suffer from constrained tool-calling trajectories. We propose a novel VideoTIR that leverages Reinforcement Learning (RL) to encourage proper usage of comprehensive multi-level toolkits for efficient long video understanding. VideoTIR explores both Zero-RL and SFT cold-starting to enable MLLMs to retrieve and focus on meaningful video segments/images/regions, enhancing long video understanding both accurately and efficiently. To reduce redundant tool-calling, we propose Toolkit Action Grouped Policy Optimization (TAGPO), which enhances the efficiency of the calling process through stepwise reward assignment and reuse of failed rollouts. Additionally, we develop a sandbox-based trajectory synthesis framework to generate high-quality trajectories data. Extensive experiments on three long-video QA benchmarks demonstrate the effectiveness and efficiency of our method.