Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence

TL;DR

Open-o3-Video introduces a framework for grounded video reasoning that explicitly highlights spatio-temporal evidence, enabling traceable and verifiable reasoning processes while achieving state-of-the-art performance on multiple benchmarks.

Contribution

The paper presents a novel non-agent framework with spatio-temporal evidence integration, a new dataset STGR, and a reinforcement learning strategy for improved video reasoning.

Findings

Achieves state-of-the-art on V-STAR with significant performance gains.

Provides verifiable reasoning traces for increased answer reliability.

Demonstrates consistent improvements across various video understanding benchmarks.

Abstract

Most video reasoning models only generate textual reasoning traces without indicating when and where key evidence appears. Recent models such as OpenAI-o3 have sparked wide interest in evidence-centered reasoning for images, yet extending this ability to videos is more challenging due to the need for joint temporal tracking and spatial localization across dynamic scenes. We introduce Open-o3-Video, a non-agent framework that integrates explicit spatio-temporal evidence into video reasoning by highlighting key timestamps, objects, and bounding boxes, making the reasoning process traceable and verifiable. To enable this capability, we first construct high-quality datasets STGR that provide unified spatio-temporal supervision, which is absent in existing resources. We further adopt a cold-start reinforcement learning strategy with specially designed rewards that jointly encourage answer accuracy, temporal alignment, and spatial precision. On the V-STAR benchmark, Open-o3-Video achieves state-of-the-art performance, improving mAM by 14.4% and mLGM by 24.2% over the Qwen2.5-VL baseline, and shows consistent gains across a range of video understanding benchmarks. Beyond accuracy, the grounded reasoning traces produced by Open-o3-Video support confidence-aware test-time scaling, improving answer reliability.