Knowledge Transfer and Domain Adaptation for Fine-Grained Remote Sensing Image Segmentation

TL;DR

This paper proposes a novel end-to-end learning framework that combines knowledge transfer and domain adaptation techniques to improve fine-grained remote sensing image segmentation, demonstrating significant performance gains on new datasets.

Contribution

It introduces the Feature Alignment Module and Feature Modulation Module for effective knowledge transfer and domain adaptation in remote sensing segmentation tasks.

Findings

Achieved 2.57 mIoU improvement on grass dataset

Achieved 3.73 mIoU improvement on cloud dataset

Demonstrated effectiveness of combined knowledge transfer and domain adaptation

Abstract

Fine-grained remote sensing image segmentation is essential for accurately identifying detailed objects in remote sensing images. Recently, vision transformer models (VTMs) pre-trained on large-scale datasets have demonstrated strong zero-shot generalization. However, directly applying them to specific tasks may lead to domain shift. We introduce a novel end-to-end learning paradigm combining knowledge guidance with domain refinement to enhance performance. We present two key components: the Feature Alignment Module (FAM) and the Feature Modulation Module (FMM). FAM aligns features from a CNN-based backbone with those from the pretrained VTM's encoder using channel transformation and spatial interpolation, and transfers knowledge via KL divergence and L2 normalization constraint. FMM further adapts the knowledge to the specific domain to address domain shift. We also introduce a fine-grained grass segmentation dataset and demonstrate, through experiments on two datasets, that our method achieves a significant improvement of 2.57 mIoU on the grass dataset and 3.73 mIoU on the cloud dataset. The results highlight the potential of combining knowledge transfer and domain adaptation to overcome domain-related challenges and data limitations. The project page is available at https://xavierjiezou.github.io/KTDA/.