Guide Local Feature Matching by Overlap Estimation

TL;DR

This paper introduces OETR, a Transformer-based overlap estimation method that improves local feature matching accuracy by focusing on the commonly visible regions between image pairs, especially under challenging conditions.

Contribution

The paper proposes a novel overlap estimation approach using Transformers, which can be integrated into existing pipelines to enhance matching performance in difficult scenarios.

Findings

Significantly improves local feature matching accuracy.

Effective for image pairs with small shared regions.

Boosts state-of-the-art performance in challenging conditions.

Abstract

Local image feature matching under large appearance, viewpoint, and distance changes is challenging yet important. Conventional methods detect and match tentative local features across the whole images, with heuristic consistency checks to guarantee reliable matches. In this paper, we introduce a novel Overlap Estimation method conditioned on image pairs with TRansformer, named OETR, to constrain local feature matching in the commonly visible region. OETR performs overlap estimation in a two-step process of feature correlation and then overlap regression. As a preprocessing module, OETR can be plugged into any existing local feature detection and matching pipeline, to mitigate potential view angle or scale variance. Intensive experiments show that OETR can boost state-of-the-art local feature matching performance substantially, especially for image pairs with small shared regions. The code will be publicly available at https://github.com/AbyssGaze/OETR.