Cross-Descriptor Visual Localization and Mapping

TL;DR

This paper introduces a novel, data-driven approach for cross-descriptor visual localization and mapping that is flexible, efficient, and compatible with various feature types, addressing limitations of traditional methods.

Contribution

It presents the first principled solution to cross-descriptor localization and mapping, enabling continuous map updates across different feature representations.

Findings

Effective across handcrafted and learned features

Scales linearly with number of feature algorithms

Low computational requirements

Abstract

Visual localization and mapping is the key technology underlying the majority of mixed reality and robotics systems. Most state-of-the-art approaches rely on local features to establish correspondences between images. In this paper, we present three novel scenarios for localization and mapping which require the continuous update of feature representations and the ability to match across different feature types. While localization and mapping is a fundamental computer vision problem, the traditional setup supposes the same local features are used throughout the evolution of a map. Thus, whenever the underlying features are changed, the whole process is repeated from scratch. However, this is typically impossible in practice, because raw images are often not stored and re-building the maps could lead to loss of the attached digital content. To overcome the limitations of current approaches, we present the first principled solution to cross-descriptor localization and mapping. Our data-driven approach is agnostic to the feature descriptor type, has low computational requirements, and scales linearly with the number of description algorithms. Extensive experiments demonstrate the effectiveness of our approach on state-of-the-art benchmarks for a variety of handcrafted and learned features.