Towards Generalizable Scene Change Detection

TL;DR

This paper introduces GeSCF, a novel framework for scene change detection that leverages the Segment Anything Model to improve generalization across unseen environments and conditions, supported by a new benchmark and dataset.

Contribution

The work presents a zero-shot, SAM-based approach for generalizable scene change detection, along with a new benchmark, metrics, and the ChangeVPR dataset for diverse environmental scenarios.

Findings

GeSCF improves performance by 19.2% on existing datasets.

Nearly doubles prior art performance on ChangeVPR.

Introduces a new benchmark and dataset for generalizable SCD.

Abstract

While current state-of-the-art Scene Change Detection (SCD) approaches achieve impressive results in well-trained research data, they become unreliable under unseen environments and different temporal conditions; in-domain performance drops from 77.6% to 8.0% in a previously unseen environment and to 4.6% under a different temporal condition -- calling for generalizable SCD and benchmark. In this work, we propose the Generalizable Scene Change Detection Framework (GeSCF), which addresses unseen domain performance and temporal consistency -- to meet the growing demand for anything SCD. Our method leverages the pre-trained Segment Anything Model (SAM) in a zero-shot manner. For this, we design Initial Pseudo-mask Generation and Geometric-Semantic Mask Matching -- seamlessly turning user-guided prompt and single-image based segmentation into scene change detection for a pair of inputs without guidance. Furthermore, we define the Generalizable Scene Change Detection (GeSCD) benchmark along with novel metrics and an evaluation protocol to facilitate SCD research in generalizability. In the process, we introduce the ChangeVPR dataset, a collection of challenging image pairs with diverse environmental scenarios -- including urban, suburban, and rural settings. Extensive experiments across various datasets demonstrate that GeSCF achieves an average performance gain of 19.2% on existing SCD datasets and 30.0% on the ChangeVPR dataset, nearly doubling the prior art performance. We believe our work can lay a solid foundation for robust and generalizable SCD research.