End-to-end learning of keypoint detection and matching for relative pose estimation

TL;DR

This paper introduces an end-to-end trainable system for keypoint detection, description, matching, and pose estimation, significantly improving visual localization accuracy for applications like AR and robotic mapping.

Contribution

It presents a novel end-to-end learning framework that integrates all steps of pose estimation, including feature matching, which was traditionally done separately.

Findings

Achieves state-of-the-art localization accuracy on 7 Scenes dataset.

Demonstrates robustness and real-time performance in visual localization tasks.

Unified learning approach simplifies the pipeline and improves overall accuracy.

Abstract

We propose a new method for estimating the relative pose between two images, where we jointly learn keypoint detection, description extraction, matching and robust pose estimation. While our architecture follows the traditional pipeline for pose estimation from geometric computer vision, all steps are learnt in an end-to-end fashion, including feature matching. We demonstrate our method for the task of visual localization of a query image within a database of images with known pose. Pairwise pose estimation has many practical applications for robotic mapping, navigation, and AR. For example, the display of persistent AR objects in the scene relies on a precise camera localization to make the digital models appear anchored to the physical environment. We train our pipeline end-to-end specifically for the problem of visual localization. We evaluate our proposed approach on localization accuracy, robustness and runtime speed. Our method achieves state of the art localization accuracy on the 7 Scenes dataset.