MTLDesc: Looking Wider to Describe Better

TL;DR

This paper introduces MTLDesc, a local descriptor learning method that incorporates global and surrounding context through attention mechanisms, significantly improving performance on various localization benchmarks.

Contribution

The paper proposes novel context augmentation modules and a spatial attention loss to enhance local descriptors with non-local awareness, surpassing prior state-of-the-art methods.

Findings

Outperforms previous local descriptors on HPatches, Aachen Day-Night, and InLoc benchmarks.

Effectively integrates global context and spatial attention into descriptor learning.

Achieves more stable and accurate keypoint localization.

Abstract

Limited by the locality of convolutional neural networks, most existing local features description methods only learn local descriptors with local information and lack awareness of global and surrounding spatial context. In this work, we focus on making local descriptors "look wider to describe better" by learning local Descriptors with More Than just Local information (MTLDesc). Specifically, we resort to context augmentation and spatial attention mechanisms to make our MTLDesc obtain non-local awareness. First, Adaptive Global Context Augmented Module and Diverse Local Context Augmented Module are proposed to construct robust local descriptors with context information from global to local. Second, Consistent Attention Weighted Triplet Loss is designed to integrate spatial attention awareness into both optimization and matching stages of local descriptors learning. Third, Local Features Detection with Feature Pyramid is given to obtain more stable and accurate keypoints localization. With the above innovations, the performance of our MTLDesc significantly surpasses the prior state-of-the-art local descriptors on HPatches, Aachen Day-Night localization and InLoc indoor localization benchmarks.