ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization

TL;DR

ZippyPoint introduces a quantized neural network for interest point detection and description that significantly accelerates inference and matching, enabling efficient use on resource-constrained devices with minimal performance loss.

Contribution

The paper proposes a novel quantization approach and binary descriptor normalization for neural networks, achieving faster and more compact interest point detection and description.

Findings

At least tenfold speedup in network inference and matching.

Significant reduction in 3D model size and computational requirements.

Minor performance degradation in localization and relocalization tasks.

Abstract

Efficient detection and description of geometric regions in images is a prerequisite in visual systems for localization and mapping. Such systems still rely on traditional hand-crafted methods for efficient generation of lightweight descriptors, a common limitation of the more powerful neural network models that come with high compute and specific hardware requirements. In this paper, we focus on the adaptations required by detection and description neural networks to enable their use in computationally limited platforms such as robots, mobile, and augmented reality devices. To that end, we investigate and adapt network quantization techniques to accelerate inference and enable its use on compute limited platforms. In addition, we revisit common practices in descriptor quantization and propose the use of a binary descriptor normalization layer, enabling the generation of distinctive binary descriptors with a constant number of ones. ZippyPoint, our efficient quantized network with binary descriptors, improves the network runtime speed, the descriptor matching speed, and the 3D model size, by at least an order of magnitude when compared to full-precision counterparts. These improvements come at a minor performance degradation as evaluated on the tasks of homography estimation, visual localization, and map-free visual relocalization. Code and models are available at https://github.com/menelaoskanakis/ZippyPoint.