Semi-supervised Multiscale Matching for SAR-Optical Image

TL;DR

This paper introduces a semi-supervised multiscale matching approach for SAR-optical image matching, leveraging both labeled and unlabeled data, with novel feature disentanglement techniques to improve accuracy and reduce manual annotation efforts.

Contribution

It proposes a semi-supervised pipeline with pseudo-labeling and a cross-modal feature enhancement module that enhances SAR-optical image matching without extensive labeled data.

Findings

Outperforms existing semi-supervised methods.

Achieves performance comparable to fully supervised state-of-the-art methods.

Demonstrates effectiveness on benchmark datasets.

Abstract

Driven by the complementary nature of optical and synthetic aperture radar (SAR) images, SAR-optical image matching has garnered significant interest. Most existing SAR-optical image matching methods aim to capture effective matching features by employing the supervision of pixel-level matched correspondences within SAR-optical image pairs, which, however, suffers from time-consuming and complex manual annotation, making it difficult to collect sufficient labeled SAR-optical image pairs. To handle this, we design a semi-supervised SAR-optical image matching pipeline that leverages both scarce labeled and abundant unlabeled image pairs and propose a semi-supervised multiscale matching for SAR-optical image matching (S2M2-SAR). Specifically, we pseudo-label those unlabeled SAR-optical image pairs with pseudo ground-truth similarity heatmaps by combining both deep and shallow level matching results, and train the matching model by employing labeled and pseudo-labeled similarity heatmaps. In addition, we introduce a cross-modal feature enhancement module trained using a cross-modality mutual independence loss, which requires no ground-truth labels. This unsupervised objective promotes the separation of modality-shared and modality-specific features by encouraging statistical independence between them, enabling effective feature disentanglement across optical and SAR modalities. To evaluate the effectiveness of S2M2-SAR, we compare it with existing competitors on benchmark datasets. Experimental results demonstrate that S2M2-SAR not only surpasses existing semi-supervised methods but also achieves performance competitive with fully supervised SOTA methods, demonstrating its efficiency and practical potential.