Learning to Make Keypoints Sub-Pixel Accurate

TL;DR

This paper introduces a neural network-based method to enhance keypoint detectors with sub-pixel accuracy, significantly improving localization precision without complex detector redesigns, and demonstrating superior performance across multiple datasets.

Contribution

A novel network that learns offset vectors for detected features, enabling sub-pixel accuracy enhancement for any detector without specialized design.

Findings

Outperforms existing methods in accuracy across datasets

Adds only ~7 ms to detection time

Effective with various matching algorithms

Abstract

This work addresses the challenge of sub-pixel accuracy in detecting 2D local features, a cornerstone problem in computer vision. Despite the advancements brought by neural network-based methods like SuperPoint and ALIKED, these modern approaches lag behind classical ones such as SIFT in keypoint localization accuracy due to their lack of sub-pixel precision. We propose a novel network that enhances any detector with sub-pixel precision by learning an offset vector for detected features, thereby eliminating the need for designing specialized sub-pixel accurate detectors. This optimization directly minimizes test-time evaluation metrics like relative pose error. Through extensive testing with both nearest neighbors matching and the recent LightGlue matcher across various real-world datasets, our method consistently outperforms existing methods in accuracy. Moreover, it adds only around 7 ms to the time of a particular detector. The code is available at https://github.com/KimSinjeong/keypt2subpx .