RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-N Ranking

TL;DR

RSRCC is a new benchmark for remote sensing change question-answering that emphasizes fine-grained semantic reasoning about localized changes, constructed via a novel semi-supervised curation pipeline.

Contribution

The paper introduces RSRCC, the first dataset focused on localized, reasoning-based change detection in remote sensing, built with a hierarchical semi-supervised pipeline using Best-of-N ranking.

Findings

RSRCC contains 126k questions for training and testing.

The dataset enables fine-grained semantic change reasoning.

A scalable curation pipeline improves data quality.

Abstract

Traditional change detection identifies where changes occur, but does not explain what changed in natural language. Existing remote sensing change captioning datasets typically describe overall image-level differences, leaving fine-grained localized semantic reasoning largely unexplored. To close this gap, we present RSRCC, a new benchmark for remote sensing change question-answering containing 126k questions, split into 87k training, 17.1k validation, and 22k test instances. Unlike prior datasets, RSRCC is built around localized, change-specific questions that require reasoning about a particular semantic change. To the best of our knowledge, this is the first remote sensing change question-answering benchmark designed explicitly for such fine-grained reasoning-based supervision. To construct RSRCC, we introduce a hierarchical semi-supervised curation pipeline that uses Best-of-N ranking as a critical final ambiguity-resolution stage. First, candidate change regions are extracted from semantic segmentation masks, then initially screened using an image-text embedding model, and finally validated through retrieval-augmented vision-language curation with Best-of-N ranking. This process enables scalable filtering of noisy and ambiguous candidates while preserving semantically meaningful changes. The dataset is available at https://huggingface.co/datasets/google/RSRCC.