PRISM: PRogressive dependency maxImization for Scale-invariant image Matching

TL;DR

PRISM introduces a progressive, scale-invariant image matching method that prunes irrelevant features and effectively handles scale discrepancies, leading to improved accuracy and efficiency in image matching tasks.

Contribution

It proposes a novel multi-scale pruning module and scale-aware dynamic attention mechanism to enhance feature relevance and scale-invariance in detector-free image matching.

Findings

Achieves state-of-the-art accuracy on multiple benchmarks.

Demonstrates superior generalization across various tasks.

Reduces matching errors by focusing on relevant features.

Abstract

Image matching aims at identifying corresponding points between a pair of images. Currently, detector-free methods have shown impressive performance in challenging scenarios, thanks to their capability of generating dense matches and global receptive field. However, performing feature interaction and proposing matches across the entire image is unnecessary, because not all image regions contribute to the matching process. Interacting and matching in unmatchable areas can introduce errors, reducing matching accuracy and efficiency. Meanwhile, the scale discrepancy issue still troubles existing methods. To address above issues, we propose PRogressive dependency maxImization for Scale-invariant image Matching (PRISM), which jointly prunes irrelevant patch features and tackles the scale discrepancy. To do this, we firstly present a Multi-scale Pruning Module (MPM) to adaptively prune irrelevant features by maximizing the dependency between the two feature sets. Moreover, we design the Scale-Aware Dynamic Pruning Attention (SADPA) to aggregate information from different scales via a hierarchical design. Our method's superior matching performance and generalization capability are confirmed by leading accuracy across various evaluation benchmarks and downstream tasks. The code is publicly available at https://github.com/Master-cai/PRISM.