Improving Local Feature Matching by Entropy-inspired Scale Adaptability and Flow-endowed Local Consistency

TL;DR

This paper introduces a novel image matching pipeline that enhances scale adaptability and local consistency, significantly improving robustness and accuracy in semi-dense image matching tasks.

Contribution

It proposes a scale-aware matching module utilizing score matrix hints and a cascaded flow refinement with a gradient loss to enforce local consistency.

Findings

Achieves robust and accurate matching performance on downstream tasks.

Effectively handles scale differences between images.

Improves local consistency of final matches.

Abstract

Recent semi-dense image matching methods have achieved remarkable success, but two long-standing issues still impair their performance. At the coarse stage, the over-exclusion issue of their mutual nearest neighbor (MNN) matching layer makes them struggle to handle cases with scale difference between images. To this end, we comprehensively revisit the matching mechanism and make a key observation that the hint concealed in the score matrix can be exploited to indicate the scale ratio. Based on this, we propose a scale-aware matching module which is exceptionally effective but introduces negligible overhead. At the fine stage, we point out that existing methods neglect the local consistency of final matches, which undermines their robustness. To this end, rather than independently predicting the correspondence for each source pixel, we reformulate the fine stage as a cascaded flow refinement problem and introduce a novel gradient loss to encourage local consistency of the flow field. Extensive experiments demonstrate that our novel matching pipeline, with these proposed modifications, achieves robust and accurate matching performance on downstream tasks.