Matching of SAR and optical images based on transformation to shared modality

TL;DR

This paper introduces a novel method for matching optical and SAR images by transforming them into a shared modality, enabling the use of existing image matching models and improving accuracy over traditional translation-based approaches.

Contribution

The authors propose a new image transformation approach to create a shared modality for optical and SAR images, facilitating better matching with pre-trained models without retraining.

Findings

Outperforms alternative image translation and matching methods.

Enables use of pre-trained models like RoMa for SAR-optical image matching.

Achieves higher matching quality and versatility on the MultiSenGE dataset.

Abstract

Significant differences in optical images and Synthetic Aperture Radar (SAR) images are caused by fundamental differences in the physical principles underlying their acquisition by Earth remote sensing platforms. These differences make precise image matching (co-registration) of these two types of images difficult. In this paper, we propose a new approach to image matching of optical and SAR images, which is based on transforming the images to a new modality. The new image modality is common to both optical and SAR images and satisfies the following conditions. First, the transformed images must have an equal pre-defined number of channels. Second, the transformed and co-registered images must be as similar as possible. Third, the transformed images must be non-degenerate, meaning they must preserve the significant features of the original images. To further match images transformed to this shared modality, we train the RoMa image matching model, which is one of the leading solutions for matching of regular digital photographs. We evaluated the proposed approach on the publicly available MultiSenGE dataset containing both optical and SAR images. We demonstrated its superiority over alternative approaches based on image translation between original modalities and various feature matching algorithms. The proposed solution not only provides better quality of matching, but is also more versatile. It enables the use of ready-made RoMa and DeDoDe models, pre-trained for regular images, without retraining for a new modality, while maintaining high-quality matching of optical and SAR images.