GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring

TL;DR

This paper introduces GMM-IKRS, a framework that refines keypoints and provides interpretable scores based on probability and localization accuracy, improving keypoint repeatability and performance in vision tasks.

Contribution

It presents a novel Gaussian Mixture Model-based method for refining keypoints and generating interpretable scores for better comparison across methods.

Findings

Improves keypoint repeatability across methods.

Enhances performance in homography and pose recovery tasks.

Consistently refines keypoints using the proposed GMM approach.

Abstract

The extraction of keypoints in images is at the basis of many computer vision applications, from localization to 3D reconstruction. Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable, making it virtually impossible to compare the quality of individual keypoints across methods. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method. Our approach leverages a modified robust Gaussian Mixture Model fit designed to both reject non-robust keypoints and refine the remaining ones. Our score comprises two components: one relates to the probability of extracting the same keypoint in an image captured from another viewpoint, the other relates to the localization accuracy of the keypoint. These two interpretable components permit a comparison of individual keypoints extracted across different methods. Through extensive experiments we demonstrate that, when applied to popular keypoint detectors, our framework consistently improves the repeatability of keypoints as well as their performance in homography and two/multiple-view pose recovery tasks.