Graph Aggregation Prototype Learning for Semantic Change Detection in Remote Sensing

TL;DR

This paper introduces GAPL-SCD, a multi-task learning framework with graph prototype aggregation to improve semantic change detection accuracy and robustness in remote sensing data.

Contribution

The paper proposes a novel multi-task learning approach with graph aggregation prototypes and feature interaction modules for enhanced semantic change detection.

Findings

Achieves state-of-the-art performance on SECOND and Landsat-SCD datasets.

Improves accuracy and robustness in complex scenes.

Effectively reduces negative transfer in multi-task learning.

Abstract

Semantic change detection (SCD) extends the binary change detection task to provide not only the change locations but also the detailed "from-to" categories in multi-temporal remote sensing data. Such detailed semantic insights into changes offer considerable advantages for a wide array of applications. However, since SCD involves the simultaneous optimization of multiple tasks, the model is prone to negative transfer due to task-specific learning difficulties and conflicting gradient flows. To address this issue, we propose Graph Aggregation Prototype Learning for Semantic Change Detection in remote sensing(GAPL-SCD). In this framework, a multi-task joint optimization method is designed to optimize the primary task of semantic segmentation and change detection, along with the auxiliary task of graph aggregation prototype learning. Adaptive weight allocation and gradient rotation methods are used to alleviate the conflict between training tasks and improve multi-task learning capabilities. Specifically, the graph aggregation prototype learning module constructs an interaction graph using high-level features. Prototypes serve as class proxies, enabling category-level domain alignment across time points and reducing interference from irrelevant changes. Additionally, the proposed self-query multi-level feature interaction and bi-temporal feature fusion modules further enhance multi-scale feature representation, improving performance in complex scenes. Experimental results on the SECOND and Landsat-SCD datasets demonstrate that our method achieves state-of-the-art performance, with significant improvements in accuracy and robustness for SCD task.