Subimage Overlap Prediction: Task-Aligned Self-Supervised Pretraining For Semantic Segmentation In Remote Sensing Imagery

TL;DR

This paper introduces Subimage Overlap Prediction, a self-supervised pretraining task that improves semantic segmentation in remote sensing imagery using less data, leading to faster convergence and better performance.

Contribution

The work proposes a novel SSL pretraining task that requires less data and enhances downstream segmentation performance in remote sensing imagery.

Findings

Pretraining with the proposed task accelerates convergence.

Achieves equal or better segmentation performance (mIoU) than existing methods.

Requires significantly less pretraining data compared to other SSL approaches.

Abstract

Self-supervised learning (SSL) methods have become a dominant paradigm for creating general purpose models whose capabilities can be transferred to downstream supervised learning tasks. However, most such methods rely on vast amounts of pretraining data. This work introduces Subimage Overlap Prediction, a novel self-supervised pretraining task to aid semantic segmentation in remote sensing imagery that uses significantly lesser pretraining imagery. Given an image, a sub-image is extracted and the model is trained to produce a semantic mask of the location of the extracted sub-image within the original image. We demonstrate that pretraining with this task results in significantly faster convergence, and equal or better performance (measured via mIoU) on downstream segmentation. This gap in convergence and performance widens when labeled training data is reduced. We show this across multiple architecture types, and with multiple downstream datasets. We also show that our method matches or exceeds performance while requiring significantly lesser pretraining data relative to other SSL methods. Code and model weights are provided at \href{https://github.com/sharmalakshay93/subimage-overlap-prediction}{github.com/sharmalakshay93/subimage-overlap-prediction}.