Not All Pixels Are Equal: Confidence-Guided Attention for Feature Matching

TL;DR

This paper introduces a confidence-guided attention mechanism for semi-dense feature matching that adaptively prunes irrelevant regions, improving discriminative feature extraction and matching accuracy.

Contribution

It proposes a novel confidence-guided attention method that uses precomputed confidence maps to refine attention weights and feature aggregation, enhancing matching performance.

Findings

Outperforms state-of-the-art methods on three benchmarks.

Effectively reduces noise from irrelevant regions during feature matching.

Improves discriminability of features through a classification loss.

Abstract

Semi-dense feature matching methods have been significantly advanced by leveraging attention mechanisms to extract discriminative descriptors. However, most existing approaches treat all pixels equally during attention computations, which can potentially introduce noise and redundancy from irrelevant regions. To address this issue, we propose a confidence-guided attention that adaptively prunes attention weights for each pixel based on precomputed matching confidence maps. These maps are generated by evaluating the mutual similarity between feature pairs extracted from the backbone, where high confidence indicates a high potential for matching. Then the attention is refined through two steps: (1) a confidence-guided bias is introduced to adaptively adjust the attention distributions for each query pixel, avoiding irrelevant interactions between non-overlap pixels; (2) the corresponding confidence map is additionally employed to rescale value features during feature aggregation, attenuating the influence of uncertain regions. Moreover, a classification loss is introduced to encourage the backbone's features to discriminate between matchable and non-matchable regions. Extensive experiments on three benchmarks demonstrate that the proposal outperforms existing state-of-the-art methods.