Training general representations for remote sensing using in-domain knowledge

TL;DR

This paper develops and evaluates general remote sensing representations using diverse datasets, demonstrating significant performance improvements in low-data scenarios and providing publicly available models and datasets.

Contribution

It introduces a systematic approach to creating and assessing generic remote sensing representations, highlighting the importance of in-domain data for transfer learning.

Findings

Including in-domain data improves performance significantly with few training samples.

Pretrained models outperform models trained from scratch or fine-tuned only on ImageNet.

The study provides publicly available datasets and pretrained models for the community.

Abstract

Automatically finding good and general remote sensing representations allows to perform transfer learning on a wide range of applications - improving the accuracy and reducing the required number of training samples. This paper investigates development of generic remote sensing representations, and explores which characteristics are important for a dataset to be a good source for representation learning. For this analysis, five diverse remote sensing datasets are selected and used for both, disjoint upstream representation learning and downstream model training and evaluation. A common evaluation protocol is used to establish baselines for these datasets that achieve state-of-the-art performance. As the results indicate, especially with a low number of available training samples a significant performance enhancement can be observed when including additionally in-domain data in comparison to training models from scratch or fine-tuning only on ImageNet (up to 11% and 40%, respectively, at 100 training samples). All datasets and pretrained representation models are published online.