StAR: Segment Anything Reasoner

TL;DR

StAR is a novel framework that enhances visual reasoning in AI by refining design choices, introducing parallel test-time scaling, and establishing a new benchmark dataset, significantly improving reasoning performance with limited training data.

Contribution

This work introduces StAR, a comprehensive reasoning framework with innovative training and testing strategies, and a new dataset for systematic evaluation of visual reasoning methods.

Findings

StAR achieves significant performance improvements over baseline models.

Parallel test-time scaling further boosts segmentation accuracy.

The approach activates latent reasoning capabilities with only 5k training samples.

Abstract

As AI systems are being integrated more rapidly into diverse and complex real-world environments, the ability to perform holistic reasoning over an implicit query and an image to localize a target is becoming increasingly important. However, recent reasoning segmentation methods fail to sufficiently elicit the visual reasoning capabilities of the base mode. In this work, we present Segment Anything Reasoner (StAR), a comprehensive framework that refines the design space from multiple perspectives-including parameter-tuning scheme, reward functions, learning strategies and answer format-and achieves substantial improvements over recent baselines. In addition, for the first time, we successfully introduce parallel test-time scaling to the segmentation task, pushing the performance boundary even further. To extend the scope and depth of reasoning covered by existing benchmark, we also construct the ReasonSeg-X, which compactly defines reasoning types and includes samples that require deeper reasoning. Leveraging this dataset, we train StAR with a rollout-expanded selective-tuning approach to activate the base model's latent reasoning capabilities, and establish a rigorous benchmark for systematic, fine-grained evaluation of advanced methods. With only 5k training samples, StAR achieves significant gains over its base counterparts across extensive benchmarks, demonstrating that our method effectively brings dormant reasoning competence to the surface.