TERDNet: Transformer Encoder-Recurrent Decoder Network for Scene Change Detection

TL;DR

TERDNet is a novel transformer-based network with recurrent decoding that significantly improves scene change detection accuracy and detail, validated on multiple benchmarks and robust to viewpoint variations.

Contribution

It introduces a transformer encoder with a recurrent decoder and fusion strategies, enhancing feature refinement and change mask precision in scene change detection.

Findings

Outperforms prior methods on four benchmarks.

Effective in handling viewpoint misalignments.

Pretraining strategies improve segmentation quality.

Abstract

In this work, we address the challenge of Scene Change Detection (SCD), where the goal is to identify variations between two images of the same location captured at different times. Existing SCD models often overlook the varying importance of features across layers, employ single-step decoders that confine refinement, and provide limited insight into encoder pretraining strategies. We propose TERDNet, a Transformer Encoder-Recurrent Decoder Network designed to overcome these limitations. TERDNet consists of a transformer-based encoder that extracts multi-level representations, a feature fusion module that integrates correlation volumes with these features, a recurrent 3-gate-GRU decoder that performs iterative refinement, and a combined convolution-interpolation upsampler that restores fine-grained resolution. Extensive experiments on four public benchmarks show that TERDNet consistently outperforms prior approaches and produces more accurate and detailed change masks. Ablation studies confirm the benefit of segmentation-based pretraining and the effectiveness of our fusion design. In addition, robustness tests under viewpoint misalignment confirm TERDNet's potential for deployment in real-world robotic systems, where reliable perception is critical. Our code is available at https://github.com/AutoCompSysLab/TERDNet.