IMP: Iterative Matching and Pose Estimation with Adaptive Pooling

TL;DR

This paper introduces IMP, an iterative framework that jointly improves feature matching and pose estimation by leveraging geometric relationships, resulting in better accuracy and efficiency in visual localization tasks.

Contribution

The paper proposes a novel geometry-aware recurrent attention module and an efficient EIMP method that iteratively refines matches and poses, reducing computational complexity.

Findings

Outperforms previous methods in accuracy on multiple datasets

Reduces quadratic time complexity of attention in transformers

Demonstrates improved efficiency and robustness in pose estimation

Abstract

Previous methods solve feature matching and pose estimation using a two-stage process by first finding matches and then estimating the pose. As they ignore the geometric relationships between the two tasks, they focus on either improving the quality of matches or filtering potential outliers, leading to limited efficiency or accuracy. In contrast, we propose an iterative matching and pose estimation framework (IMP) leveraging the geometric connections between the two tasks: a few good matches are enough for a roughly accurate pose estimation; a roughly accurate pose can be used to guide the matching by providing geometric constraints. To this end, we implement a geometry-aware recurrent attention-based module which jointly outputs sparse matches and camera poses. Specifically, for each iteration, we first implicitly embed geometric information into the module via a pose-consistency loss, allowing it to predict geometry-aware matches progressively. Second, we introduce an \textbf{e}fficient IMP, called EIMP, to dynamically discard keypoints without potential matches, avoiding redundant updating and significantly reducing the quadratic time complexity of attention computation in transformers. Experiments on YFCC100m, Scannet, and Aachen Day-Night datasets demonstrate that the proposed method outperforms previous approaches in terms of accuracy and efficiency.