Land Use Prediction using Electro-Optical to SAR Few-Shot Transfer Learning

TL;DR

This paper explores transfer learning between electro-optical and SAR satellite images for land use prediction, demonstrating improved accuracy in few-shot scenarios through novel training techniques and embedding space shaping.

Contribution

It introduces extensions to sliced Wasserstein distance-based transfer learning for satellite imagery, enhancing stability and performance in practical land use classification tasks.

Findings

Outperforms baseline models in few-shot LCZ prediction

Instance normalization stabilizes training process

Supervised contrastive learning improves embedding quality

Abstract

Satellite image analysis has important implications for land use, urbanization, and ecosystem monitoring. Deep learning methods can facilitate the analysis of different satellite modalities, such as electro-optical (EO) and synthetic aperture radar (SAR) imagery, by supporting knowledge transfer between the modalities to compensate for individual shortcomings. Recent progress has shown how distributional alignment of neural network embeddings can produce powerful transfer learning models by employing a sliced Wasserstein distance (SWD) loss. We analyze how this method can be applied to Sentinel-1 and -2 satellite imagery and develop several extensions toward making it effective in practice. In an application to few-shot Local Climate Zone (LCZ) prediction, we show that these networks outperform multiple common baselines on datasets with a large number of classes. Further, we provide evidence that instance normalization can significantly stabilize the training process and that explicitly shaping the embedding space using supervised contrastive learning can lead to improved performance.