Detect Changes like Humans: Incorporating Semantic Priors for Improved Change Detection

TL;DR

This paper introduces SA-CDNet, a novel change detection model that incorporates semantic priors from foundation models like FastSAM, improving accuracy by mimicking human visual comparison and effectively handling noise and illumination variations.

Contribution

The paper proposes a semantic-aware change detection network that transfers foundation model knowledge and introduces a pre-training strategy using pseudo-change data from single-temporal segmentation datasets.

Findings

Outperforms state-of-the-art methods on five benchmarks.

Effective integration of semantic priors enhances change detection accuracy.

Pre-training with pseudo-change data improves model adaptation.

Abstract

When given two similar images, humans identify their differences by comparing the appearance (e.g., color, texture) with the help of semantics (e.g., objects, relations). However, mainstream binary change detection models adopt a supervised training paradigm, where the annotated binary change map is the main constraint. Thus, such methods primarily emphasize difference-aware features between bi-temporal images, and the semantic understanding of changed landscapes is undermined, resulting in limited accuracy in the face of noise and illumination variations. To this end, this paper explores incorporating semantic priors from visual foundation models to improve the ability to detect changes. Firstly, we propose a Semantic-Aware Change Detection network (SA-CDNet), which transfers the knowledge of visual foundation models (i.e., FastSAM) to change detection. Inspired by the human visual paradigm, a novel dual-stream feature decoder is derived to distinguish changes by combining semantic-aware features and difference-aware features. Secondly, we explore a single-temporal pre-training strategy for better adaptation of visual foundation models. With pseudo-change data constructed from single-temporal segmentation datasets, we employ an extra branch of proxy semantic segmentation task for pre-training. We explore various settings like dataset combinations and landscape types, thus providing valuable insights. Experimental results on five challenging benchmarks demonstrate the superiority of our method over the existing state-of-the-art methods. The code is available at $\href{https://github.com/DREAMXFAR/SA-CDNet}{github}$.