A Contrastive Learning Foundation Model Based on Perfectly Aligned Sample Pairs for Remote Sensing Images

TL;DR

This paper introduces PerA, a contrastive self-supervised learning method for remote sensing images that uses perfectly aligned sample pairs to produce high-quality, adaptable features with improved memory efficiency and semantic consistency.

Contribution

The paper proposes a novel contrastive learning framework, PerA, utilizing semantically perfectly aligned sample pairs and masked views to enhance remote sensing image feature extraction.

Findings

Achieves comparable performance to state-of-the-art methods on downstream tasks.

Demonstrates high memory efficiency and scalability with larger batch sizes.

Shows robustness and adaptability to uncurated remote sensing data.

Abstract

Self-Supervised Learning (SSL) enables us to pre-train foundation models without costly labeled data. Among SSL methods, Contrastive Learning (CL) methods are better at obtaining accurate semantic representations in noise interference. However, due to the significant domain gap, while CL methods have achieved great success in many computer vision tasks, they still require specific adaptation for Remote Sensing (RS) images. To this end, we present a novel self-supervised method called PerA, which produces all-purpose RS features through semantically Perfectly Aligned sample pairs. Specifically, PerA obtains features from sampled views by applying spatially disjoint masks to augmented images rather than random cropping. Our framework provides high-quality features by ensuring consistency between teacher and student and predicting learnable mask tokens. Compared to previous contrastive methods, our method demonstrates higher memory efficiency and can be trained with larger batches due to its sparse inputs. Additionally, the proposed method demonstrates remarkable adaptability to uncurated RS data and reduce the impact of the potential semantic inconsistency. We also collect an unlabeled pre-training dataset, which contains about 5 million RS images. We conducted experiments on multiple downstream task datasets and achieved performance comparable to previous state-of-the-art methods with a limited model scale, demonstrating the effectiveness of our approach. We hope this work will contribute to practical remote sensing interpretation works.