# GS: Generative Segmentation via Label Diffusion

## TL;DR

GS introduces a novel generative framework for language-driven image segmentation by directly generating segmentation masks from noise conditioned on images and language, outperforming existing methods on challenging benchmarks.

## Contribution

The paper proposes a new generative segmentation approach that reverses traditional diffusion processes, enabling end-to-end training and improved segmentation quality.

## Key findings

- Outperforms existing methods on Panoptic Narrative Grounding benchmark
- Sets a new state-of-the-art in language-driven segmentation
- Demonstrates effective control over spatial and semantic fidelity

## Abstract

Language-driven image segmentation is a fundamental task in vision-language understanding, requiring models to segment regions of an image corresponding to natural language expressions. Traditional methods approach this as a discriminative problem, assigning each pixel to foreground or background based on semantic alignment. Recently, diffusion models have been introduced to this domain, but existing approaches remain image-centric: they either (i) use image diffusion models as visual feature extractors, (ii) synthesize segmentation data via image generation to train discriminative models, or (iii) perform diffusion inversion to extract attention cues from pre-trained image diffusion models-thereby treating segmentation as an auxiliary process. In this paper, we propose GS (Generative Segmentation), a novel framework that formulates segmentation itself as a generative task via label diffusion. Instead of generating images conditioned on label maps and text, GS reverses the generative process: it directly generates segmentation masks from noise, conditioned on both the input image and the accompanying language description. This paradigm makes label generation the primary modeling target, enabling end-to-end training with explicit control over spatial and semantic fidelity. To demonstrate the effectiveness of our approach, we evaluate GS on Panoptic Narrative Grounding (PNG), a representative and challenging benchmark for multimodal segmentation that requires panoptic-level reasoning guided by narrative captions. Experimental results show that GS significantly outperforms existing discriminative and diffusion-based methods, setting a new state-of-the-art for language-driven segmentation.