LaVIDE: A Language-Vision Discriminator for Detecting Changes in Satellite Image with Map References

TL;DR

This paper introduces LaVIDE, a novel language-vision discriminator that effectively detects changes in satellite images by leveraging map references and high-level semantic information, outperforming existing methods.

Contribution

LaVIDE bridges the gap between map categories and satellite images using language-vision models, introducing a mixture-of-experts module for comprehensive semantic comparison in change detection.

Findings

Outperforms state-of-the-art algorithms on benchmark datasets

Achieves 13.8% improvement on DynamicEarthNet

Achieves 4.3% improvement on SECOND dataset

Abstract

Change detection, which typically relies on the comparison of bi-temporal images, is significantly hindered when only a single image is available. Comparing a single image with an existing map, such as OpenStreetMap, which is continuously updated through crowd-sourcing, offers a viable solution to this challenge. Unlike images that carry low-level visual details of ground objects, maps convey high-level categorical information. This discrepancy in abstraction levels complicates the alignment and comparison of the two data types. In this paper, we propose a \textbf{La}nguage-\textbf{VI}sion \textbf{D}iscriminator for d\textbf{E}tecting changes in satellite image with map references, namely \ours{}, which leverages language to bridge the information gap between maps and images. Specifically, \ours{} formulates change detection as the problem of ``{\textit Does the pixel belong to [class]?}'', aligning maps and images within the feature space of the language-vision model to associate high-level map categories with low-level image details. Moreover, we build a mixture-of-experts discriminative module, which compares linguistic features from maps with visual features from images across various semantic perspectives, achieving comprehensive semantic comparison for change detection. Extensive evaluation on four benchmark datasets demonstrates that \ours{} can effectively detect changes in satellite image with map references, outperforming state-of-the-art change detection algorithms, e.g., with gains of about $13.8$\% on the DynamicEarthNet dataset and $4.3$\% on the SECOND dataset.