VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learning

TL;DR

VideoSeg-R1 introduces a reinforcement learning framework for video object segmentation that enhances generalization and explicit reasoning, outperforming existing methods on multiple benchmarks.

Contribution

It is the first to incorporate reinforcement learning into video reasoning segmentation with a decoupled architecture and adaptive reasoning control.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively models explicit reasoning chains.

Improves generalization to out-of-distribution scenarios.

Abstract

Traditional video reasoning segmentation methods rely on supervised fine-tuning, which limits generalization to out-of-distribution scenarios and lacks explicit reasoning. To address this, we propose \textbf{VideoSeg-R1}, the first framework to introduce reinforcement learning into video reasoning segmentation. It adopts a decoupled architecture that formulates the task as joint referring image segmentation and video mask propagation. It comprises three stages: (1) A hierarchical text-guided frame sampler to emulate human attention; (2) A reasoning model that produces spatial cues along with explicit reasoning chains; and (3) A segmentation-propagation stage using SAM2 and XMem. A task difficulty-aware mechanism adaptively controls reasoning length for better efficiency and accuracy. Extensive evaluations on multiple benchmarks demonstrate that VideoSeg-R1 achieves state-of-the-art performance in complex video reasoning and segmentation tasks. The code will be publicly available at https://github.com/euyis1019/VideoSeg-R1.