EpiMask: Leveraging Epipolar Distance Based Masks in Cross-Attention for Satellite Image Matching

TL;DR

EpiMask is a novel satellite image matching network that integrates epipolar geometry constraints into the attention mechanism, significantly improving matching accuracy over traditional ground-based models.

Contribution

The paper introduces EpiMask, which incorporates epipolar distance-based masks and affine camera approximations into a deep learning framework for satellite image matching.

Findings

Up to 30% improvement in matching accuracy on SatDepth dataset

Effective integration of epipolar geometry into attention mechanisms

Enhanced robustness of feature extraction for satellite images

Abstract

The deep-learning based image matching networks can now handle significantly larger variations in viewpoints and illuminations while providing matched pairs of pixels with sub-pixel precision. These networks have been trained with ground-based image datasets and, implicitly, their performance is optimized for the pinhole camera geometry. Consequently, you get suboptimal performance when such networks are used to match satellite images since those images are synthesized as a moving satellite camera records one line at a time of the points on the ground. In this paper, we present EpiMask, a semi-dense image matching network for satellite images that (1) Incorporates patch-wise affine approximations to the camera modeling geometry; (2) Uses an epipolar distance-based attention mask to restrict cross-attention to geometrically plausible regions; and (3) That fine-tunes a foundational pretrained image encoder for robust feature extraction. Experiments on the SatDepth dataset demonstrate up to 30% improvement in matching accuracy compared to re-trained ground-based models.