A Streamlined Attention-Based Network for Descriptor Extraction

TL;DR

SANDesc is a lightweight, attention-based neural network for keypoint descriptor extraction that improves matching accuracy across multiple datasets without altering keypoint detection, using an efficient Residual U-Net architecture.

Contribution

The paper introduces SANDesc, a novel attention-enhanced network architecture for descriptor extraction, and a new urban dataset for evaluating feature extractors.

Findings

SANDesc outperforms existing descriptors on multiple benchmarks.

The model achieves high accuracy with only 2.4 million parameters.

The new urban dataset enables comprehensive evaluation of feature extractors.

Abstract

We introduce SANDesc, a Streamlined Attention-Based Network for Descriptor extraction that aims to improve on existing architectures for keypoint description. Our descriptor network learns to compute descriptors that improve matching without modifying the underlying keypoint detector. We employ a revised U-Net-like architecture enhanced with Convolutional Block Attention Modules and residual paths, enabling effective local representation while maintaining computational efficiency. We refer to the building blocks of our model as Residual U-Net Blocks with Attention. The model is trained using a modified triplet loss in combination with a curriculum learning-inspired hard negative mining strategy, which improves training stability. Extensive experiments on HPatches, MegaDepth-1500, and the Image Matching Challenge 2021 show that training SANDesc on top of existing keypoint detectors leads to improved results on multiple matching tasks compared to the original keypoint descriptors. At the same time, SANDesc has a model complexity of just 2.4 million parameters. As a further contribution, we introduce a new urban dataset featuring 4K images and pre-calibrated intrinsics, designed to evaluate feature extractors. On this benchmark, SANDesc achieves substantial performance gains over the existing descriptors while operating with limited computational resources.