S4: Self-Supervised Sensing Across the Spectrum

TL;DR

S4 introduces a self-supervised pre-training method for satellite image time series segmentation, leveraging multi-spectral data and spatial alignment to reduce labeled data needs and improve performance.

Contribution

The paper presents S4, a novel self-supervised pre-training approach utilizing multi-spectral satellite data and spatial alignment, along with a large unlabeled dataset m2s2-SITS for improved segmentation.

Findings

S4 outperforms baseline models with limited labeled data.

Pre-training with m2s2-SITS enhances segmentation accuracy.

S4 reduces the need for extensive annotations in satellite image analysis.

Abstract

Satellite image time series (SITS) segmentation is crucial for many applications like environmental monitoring, land cover mapping and agricultural crop type classification. However, training models for SITS segmentation remains a challenging task due to the lack of abundant training data, which requires fine grained annotation. We propose S4 a new self-supervised pre-training approach that significantly reduces the requirement for labeled training data by utilizing two new insights: (a) Satellites capture images in different parts of the spectrum such as radio frequencies, and visible frequencies. (b) Satellite imagery is geo-registered allowing for fine-grained spatial alignment. We use these insights to formulate pre-training tasks in S4. We also curate m2s2-SITS, a large-scale dataset of unlabeled, spatially-aligned, multi-modal and geographic specific SITS that serves as representative pre-training data for S4. Finally, we evaluate S4 on multiple SITS segmentation datasets and demonstrate its efficacy against competing baselines while using limited labeled data.