GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

TL;DR

GRAD-Former introduces a novel, efficient transformer-based framework with adaptive modules for improved change detection in high-resolution satellite imagery, outperforming existing methods across multiple datasets.

Contribution

The paper proposes GRAD-Former, a new change detection model with adaptive feature relevance modules that enhance contextual understanding while reducing computational complexity.

Findings

Outperforms existing models on LEVIR-CD, CDD, DSIFN-CD datasets

Uses fewer parameters than state-of-the-art approaches

Establishes new performance benchmarks in remote sensing change detection

Abstract

Change detection (CD) in remote sensing aims to identify semantic differences between satellite images captured at different times. While deep learning has significantly advanced this field, existing approaches based on convolutional neural networks (CNNs), transformers and Selective State Space Models (SSMs) still struggle to precisely delineate change regions. In particular, traditional transformer-based methods suffer from quadratic computational complexity when applied to very high-resolution (VHR) satellite images and often perform poorly with limited training data, leading to under-utilization of the rich spatial information available in VHR imagery. We present GRAD-Former, a novel framework that enhances contextual understanding while maintaining efficiency through reduced model size. The proposed framework consists of a novel encoder with Adaptive Feature Relevance and Refinement (AFRAR) module, fusion and decoder blocks. AFRAR integrates global-local contextual awareness through two proposed components: the Selective Embedding Amplification (SEA) module and the Global-Local Feature Refinement (GLFR) module. SEA and GLFR leverage gating mechanisms and differential attention, respectively, which generates multiple softmax heaps to capture important features while minimizing the captured irreverent features. Multiple experiments across three challenging CD datasets (LEVIR-CD, CDD, DSIFN-CD) demonstrate GRAD-Former's superior performance compared to existing approaches. Notably, GRAD-Former outperforms the current state-of-the-art models across all the metrics and all the datasets while using fewer parameters. Our framework establishes a new benchmark for remote sensing change detection performance. Our code will be released at: https://github.com/Ujjwal238/GRAD-Former