ASLFeat: Learning Local Features of Accurate Shape and Localization

TL;DR

ASLFeat introduces a novel approach to local feature detection and description by incorporating shape-awareness and precise localization, significantly improving geometric invariance and accuracy in tasks like 3D reconstruction.

Contribution

The paper proposes three lightweight modifications using deformable convolutions, feature hierarchy, and peakiness measures to enhance local feature detection and description.

Findings

Achieves state-of-the-art results across various scenarios.

Improves geometric invariance and localization accuracy.

Demonstrates effectiveness of proposed modifications.

Abstract

This work focuses on mitigating two limitations in the joint learning of local feature detectors and descriptors. First, the ability to estimate the local shape (scale, orientation, etc.) of feature points is often neglected during dense feature extraction, while the shape-awareness is crucial to acquire stronger geometric invariance. Second, the localization accuracy of detected keypoints is not sufficient to reliably recover camera geometry, which has become the bottleneck in tasks such as 3D reconstruction. In this paper, we present ASLFeat, with three light-weight yet effective modifications to mitigate above issues. First, we resort to deformable convolutional networks to densely estimate and apply local transformation. Second, we take advantage of the inherent feature hierarchy to restore spatial resolution and low-level details for accurate keypoint localization. Finally, we use a peakiness measurement to relate feature responses and derive more indicative detection scores. The effect of each modification is thoroughly studied, and the evaluation is extensively conducted across a variety of practical scenarios. State-of-the-art results are reported that demonstrate the superiority of our methods.