SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised Pretraining

TL;DR

SenPa-MAE is a transformer-based model that encodes sensor parameters for multispectral satellite imagery, enabling effective pretraining across diverse sensors and fostering sensor-independent Earth observation applications.

Contribution

Introduces a sensor parameter encoding module and data augmentation strategy, allowing pretraining on multispectral data from different satellites with non-matching sensor characteristics.

Findings

Enables cross-sensor pretraining and inference.

Improves differentiation between sensor types.

Facilitates sensor-independent Earth observation models.

Abstract

This paper introduces SenPa-MAE, a transformer architecture that encodes the sensor parameters of an observed multispectral signal into the image embeddings. SenPa-MAE can be pre-trained on imagery of different satellites with non-matching spectral or geometrical sensor characteristics. To incorporate sensor parameters, we propose a versatile sensor parameter encoding module as well as a data augmentation strategy for the diversification of the pre-training dataset. This enables the model to effectively differentiate between various sensors and gain an understanding of sensor parameters and the correlation to the observed signal. Given the rising number of Earth observation satellite missions and the diversity in their sensor specifications, our approach paves the way towards a sensor-independent Earth observation foundation model. This opens up possibilities such as cross-sensor training and sensor-independent inference.