FUSELOC: Fusing Global and Local Descriptors to Disambiguate 2D-3D Matching in Visual Localization

TL;DR

FUSELOC introduces a novel method that combines global and local descriptors through a weighted averaging technique, significantly improving 2D-3D matching accuracy in visual localization while reducing memory usage and increasing speed.

Contribution

The paper presents a new descriptor fusion approach that enhances direct matching accuracy and efficiency, bridging the gap between local-only and hierarchical localization methods.

Findings

Achieves accuracy close to hierarchical methods

Uses 43% less memory than hierarchical approaches

Runs 1.6 times faster than local-only systems

Abstract

Hierarchical visual localization methods achieve state-of-the-art accuracy but require substantial memory as they need to store all database images. Direct 2D-3D matching requires significantly less memory but suffers from lower accuracy due to the larger and more ambiguous search space. We address this ambiguity by fusing local and global descriptors using a weighted average operator. This operator rearranges the local descriptor space so that geographically nearby local descriptors are closer in the feature space according to the global descriptors. This decreases the number of irrelevant competing descriptors, especially if they are geographically distant, thus increasing the correct matching likelihood. We consistently improve the accuracy over local-only systems, and we achieve performance close to hierarchical methods while using 43\% less memory and running 1.6 times faster. Extensive experiments on four challenging datasets -- Cambridge Landmarks, Aachen Day/Night, RobotCar Seasons, and Extended CMU Seasons -- demonstrate that, for the first time, direct matching algorithms can benefit from global descriptors without compromising computational efficiency. Our code is available at \href{https://github.com/sontung/descriptor-disambiguation}{https://github.com/sontung/descriptor-disambiguation}.