Learning to Guide Local Feature Matches

TL;DR

This paper introduces a learning-based method to improve local feature matching accuracy by guiding keypoint correspondences with approximate image matching, enhancing existing descriptors and benefiting various computer vision tasks.

Contribution

It presents a novel approach that uses weak supervision from epipolar geometry to improve keypoint matching, outperforming stronger supervision methods and boosting existing descriptors.

Findings

Improves SIFT to match deep descriptors like Superpoint and D2-Net.

Weak supervision from epipolar geometry outperforms point-level supervision.

Enhances localization and 3D reconstruction in challenging conditions.

Abstract

We tackle the problem of finding accurate and robust keypoint correspondences between images. We propose a learning-based approach to guide local feature matches via a learned approximate image matching. Our approach can boost the results of SIFT to a level similar to state-of-the-art deep descriptors, such as Superpoint, ContextDesc, or D2-Net and can improve performance for these descriptors. We introduce and study different levels of supervision to learn coarse correspondences. In particular, we show that weak supervision from epipolar geometry leads to performances higher than the stronger but more biased point level supervision and is a clear improvement over weak image level supervision. We demonstrate the benefits of our approach in a variety of conditions by evaluating our guided keypoint correspondences for localization of internet images on the YFCC100M dataset and indoor images on theSUN3D dataset, for robust localization on the Aachen day-night benchmark and for 3D reconstruction in challenging conditions using the LTLL historical image data.