ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing

TL;DR

ChangeMinds is a unified multi-task framework that simultaneously performs change detection and change captioning in remote sensing images, leveraging a novel ChangeLSTM module and cross-attention to improve efficiency and accuracy.

Contribution

The paper introduces ChangeMinds, the first end-to-end multi-task model for concurrent change detection and captioning, with a new ChangeLSTM module and cross-attention mechanism.

Findings

Outperforms existing methods on LEVIR-MCI and other benchmarks.

Effectively captures complex spatiotemporal dynamics for both tasks.

Enhances multi-task learning efficiency and accuracy.

Abstract

Recent advancements in Remote Sensing (RS) for Change Detection (CD) and Change Captioning (CC) have seen substantial success by adopting deep learning techniques. Despite these advances, existing methods often handle CD and CC tasks independently, leading to inefficiencies from the absence of synergistic processing. In this paper, we present ChangeMinds, a novel unified multi-task framework that concurrently optimizes CD and CC processes within a single, end-to-end model. We propose the change-aware long short-term memory module (ChangeLSTM) to effectively capture complex spatiotemporal dynamics from extracted bi-temporal deep features, enabling the generation of universal change-aware representations that effectively serve both CC and CD tasks. Furthermore, we introduce a multi-task predictor with a cross-attention mechanism that enhances the interaction between image and text features, promoting efficient simultaneous learning and processing for both tasks. Extensive evaluations on the LEVIR-MCI dataset, alongside other standard benchmarks, show that ChangeMinds surpasses existing methods in multi-task learning settings and markedly improves performance in individual CD and CC tasks. Codes and pre-trained models will be available online.