SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images

TL;DR

This paper introduces LaSeRS, a large-scale dataset for complex language-guided segmentation in remote sensing images, and proposes SegEarth-R2, a new model architecture that effectively handles multi-target, hierarchical, and reasoning-based segmentation tasks.

Contribution

The paper presents LaSeRS, a comprehensive dataset for complex geospatial segmentation, and SegEarth-R2, a novel model architecture with spatial attention and flexible query mechanisms for improved performance.

Findings

SegEarth-R2 outperforms existing models on LaSeRS and other benchmarks.

The spatial attention supervision improves localization of small objects.

The flexible segmentation query mechanism handles multi-target scenarios effectively.

Abstract

Effectively grounding complex language to pixels in remote sensing (RS) images is a critical challenge for applications like disaster response and environmental monitoring. Current models can parse simple, single-target commands but fail when presented with complex geospatial scenarios, e.g., segmenting objects at various granularities, executing multi-target instructions, and interpreting implicit user intent. To drive progress against these failures, we present LaSeRS, the first large-scale dataset built for comprehensive training and evaluation across four critical dimensions of language-guided segmentation: hierarchical granularity, target multiplicity, reasoning requirements, and linguistic variability. By capturing these dimensions, LaSeRS moves beyond simple commands, providing a benchmark for complex geospatial reasoning. This addresses a critical gap: existing datasets oversimplify, leading to sensitivity-prone real-world models. We also propose SegEarth-R2, an MLLM architecture designed for comprehensive language-guided segmentation in RS, which directly confronts these challenges. The model's effectiveness stems from two key improvements: (1) a spatial attention supervision mechanism specifically handles the localization of small objects and their components, and (2) a flexible and efficient segmentation query mechanism that handles both single-target and multi-target scenarios. Experimental results demonstrate that our SegEarth-R2 achieves outstanding performance on LaSeRS and other benchmarks, establishing a powerful baseline for the next generation of geospatial segmentation. All data and code will be released at https://github.com/earth-insights/SegEarth-R2.