Improving the matching of deformable objects by learning to detect keypoints

TL;DR

This paper introduces a learned keypoint detection method that improves non-rigid image matching accuracy by training a CNN with geometric and photometric warpings, outperforming existing detectors.

Contribution

A novel end-to-end CNN-based keypoint detector trained with geometric and photometric warpings to enhance non-rigid image correspondence.

Findings

Increases correct matches for non-rigid images

Outperforms state-of-the-art keypoint detectors by 20 percentage points

Performs competitively in object retrieval tasks

Abstract

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. For that, we apply geometric and photometric warpings to images to generate a supervisory signal, allowing the optimization of the detector. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023