A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmark

TL;DR

This paper introduces NWPU-Refer, the largest diverse dataset for referring remote sensing image segmentation, and proposes MRSNet, a novel network architecture that achieves state-of-the-art results on this challenging task.

Contribution

The paper presents NWPU-Refer, a large-scale, diverse dataset for RRSIS, and introduces MRSNet, a new framework with innovative modules for improved segmentation performance.

Findings

MRSNet outperforms existing methods on NWPU-Refer dataset.

The dataset covers 30+ countries with high-resolution images.

The proposed modules effectively capture fine details and cross-scale features.

Abstract

Referring Remote Sensing Image Segmentation is a complex and challenging task that integrates the paradigms of computer vision and natural language processing. Existing datasets for RRSIS suffer from critical limitations in resolution, scene diversity, and category coverage, which hinders the generalization and real-world applicability of refer segmentation models. To facilitate the development of this field, we introduce NWPU-Refer, the largest and most diverse RRSIS dataset to date, comprising 15,003 high-resolution images (1024-2048px) spanning 30+ countries with 49,745 annotated targets supporting single-object, multi-object, and non-object segmentation scenarios. Additionally, we propose the Multi-scale Referring Segmentation Network (MRSNet), a novel framework tailored for the unique demands of RRSIS. MRSNet introduces two key innovations: (1) an Intra-scale Feature Interaction Module (IFIM) that captures fine-grained details within each encoder stage, and (2) a Hierarchical Feature Interaction Module (HFIM) to enable seamless cross-scale feature fusion, preserving spatial integrity while enhancing discriminative power. Extensive experiments conducte on the proposed NWPU-Refer dataset demonstrate that MRSNet achieves state-of-the-art performance across multiple evaluation metrics, validating its effectiveness. The dataset and code are publicly available at https://github.com/CVer-Yang/NWPU-Refer.