Reasoning Segmentation for Images and Videos: A Survey

TL;DR

This survey reviews the emerging field of Reasoning Segmentation, which combines visual perception with natural language reasoning to improve image and video object delineation, highlighting methods, datasets, and future challenges.

Contribution

It provides the first comprehensive overview of Reasoning Segmentation techniques, datasets, evaluation metrics, and applications, offering insights into current research gaps and future directions.

Findings

Reviewed 26 state-of-the-art RS methods

Analyzed 29 datasets and benchmarks

Identified key research gaps and future opportunities

Abstract

Reasoning Segmentation (RS) aims to delineate objects based on implicit text queries, the interpretation of which requires reasoning and knowledge integration. Unlike the traditional formulation of segmentation problems that relies on fixed semantic categories or explicit prompting, RS bridges the gap between visual perception and human-like reasoning capabilities, facilitating more intuitive human-AI interaction through natural language. Our work presents the first comprehensive survey of RS for image and video processing, examining 26 state-of-the-art methods together with a review of the corresponding evaluation metrics, as well as 29 datasets and benchmarks. We also explore existing applications of RS across diverse domains and identify their potential extensions. Finally, we identify current research gaps and highlight promising future directions.