Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series

TL;DR

This paper introduces ALISE, a novel encoder for satellite image time series that effectively handles irregular and unaligned data by producing aligned representations, improving downstream tasks like segmentation and change detection.

Contribution

ALISE is a new model that leverages spectral, spatial, and temporal data with a flexible query mechanism and multi-view framework, addressing key challenges in real-world satellite imagery applications.

Findings

ALISE outperforms previous SSL methods in segmentation tasks.

Aligned representations improve linear probing accuracy.

Change detection can be performed without supervision.

Abstract

Although recently several foundation models for satellite remote sensing imagery have been proposed, they fail to address major challenges of real/operational applications. Indeed, embeddings that don't take into account the spectral, spatial and temporal dimensions of the data as well as the irregular or unaligned temporal sampling are of little use for most real world uses. As a consequence, we propose an ALIgned Sits Encoder (ALISE), a novel approach that leverages the spatial, spectral, and temporal dimensions of irregular and unaligned SITS while producing aligned latent representations. Unlike SSL models currently available for SITS, ALISE incorporates a flexible query mechanism to project the SITS into a common and learned temporal projection space. Additionally, thanks to a multi-view framework, we explore integration of instance discrimination along a masked autoencoding task to SITS. The quality of the produced representation is assessed through three downstream tasks: crop segmentation (PASTIS), land cover segmentation (MultiSenGE), and a novel crop change detection dataset. Furthermore, the change detection task is performed without supervision. The results suggest that the use of aligned representations is more effective than previous SSL methods for linear probing segmentation tasks.