Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters

TL;DR

Key.Net is a novel keypoint detection method combining handcrafted and learned CNN filters in a multi-scale architecture, achieving superior repeatability and matching performance on benchmark datasets.

Contribution

The paper introduces a hybrid CNN architecture with handcrafted and learned filters, and a scale-aware loss function for robust keypoint detection across scales.

Findings

Outperforms state-of-the-art detectors in repeatability

Achieves higher matching performance

Has lower complexity

Abstract

We introduce a novel approach for keypoint detection task that combines handcrafted and learned CNN filters within a shallow multi-scale architecture. Handcrafted filters provide anchor structures for learned filters, which localize, score and rank repeatable features. Scale-space representation is used within the network to extract keypoints at different levels. We design a loss function to detect robust features that exist across a range of scales and to maximize the repeatability score. Our Key.Net model is trained on data synthetically created from ImageNet and evaluated on HPatches benchmark. Results show that our approach outperforms state-of-the-art detectors in terms of repeatability, matching performance and complexity.