The color out of space: learning self-supervised representations for Earth Observation imagery

TL;DR

This paper introduces a self-supervised learning approach for Earth Observation imagery by using spectral band-based colorization, improving land cover classification and disease detection without large annotated datasets.

Contribution

It proposes a novel colorization pretext task leveraging spectral bands for self-supervised learning in satellite imagery, outperforming traditional transfer learning from natural images.

Findings

Colorization improves feature extraction for satellite images.

Ensemble models combining natural image and colorization methods outperform individual techniques.

Self-supervised representations enhance land cover classification and disease detection.

Abstract

The recent growth in the number of satellite images fosters the development of effective deep-learning techniques for Remote Sensing (RS). However, their full potential is untapped due to the lack of large annotated datasets. Such a problem is usually countered by fine-tuning a feature extractor that is previously trained on the ImageNet dataset. Unfortunately, the domain of natural images differs from the RS one, which hinders the final performance. In this work, we propose to learn meaningful representations from satellite imagery, leveraging its high-dimensionality spectral bands to reconstruct the visible colors. We conduct experiments on land cover classification (BigEarthNet) and West Nile Virus detection, showing that colorization is a solid pretext task for training a feature extractor. Furthermore, we qualitatively observe that guesses based on natural images and colorization rely on different parts of the input. This paves the way to an ensemble model that eventually outperforms both the above-mentioned techniques.