Remote Sensing Scene Classification with Masked Image Modeling (MIM)

TL;DR

This paper demonstrates that Masked Image Modeling (MIM) pretraining significantly improves remote sensing scene classification accuracy using Vision Transformers, outperforming supervised learning methods and rivaling specialized models.

Contribution

It is the first to systematically evaluate MIM pretraining for remote sensing scene classification, showing substantial performance gains over supervised learning and competitive results with specialized models.

Findings

MIM-pretrained ViTs outperform supervised counterparts by up to 5% accuracy.

MIM pretraining improves accuracy by up to 18% on top-1 metrics.

MIM-pretrained ViTs achieve performance comparable to specialized Transformer models.

Abstract

Remote sensing scene classification has been extensively studied for its critical roles in geological survey, oil exploration, traffic management, earthquake prediction, wildfire monitoring, and intelligence monitoring. In the past, the Machine Learning (ML) methods for performing the task mainly used the backbones pretrained in the manner of supervised learning (SL). As Masked Image Modeling (MIM), a self-supervised learning (SSL) technique, has been shown as a better way for learning visual feature representation, it presents a new opportunity for improving ML performance on the scene classification task. This research aims to explore the potential of MIM pretrained backbones on four well-known classification datasets: Merced, AID, NWPU-RESISC45, and Optimal-31. Compared to the published benchmarks, we show that the MIM pretrained Vision Transformer (ViTs) backbones outperform other alternatives (up to 18% on top 1 accuracy) and that the MIM technique can learn better feature representation than the supervised learning counterparts (up to 5% on top 1 accuracy). Moreover, we show that the general-purpose MIM-pretrained ViTs can achieve competitive performance as the specially designed yet complicated Transformer for Remote Sensing (TRS) framework. Our experiment results also provide a performance baseline for future studies.