Retrieval and Localization with Observation Constraints

TL;DR

The paper introduces RLOCS, a visual re-localization method combining retrieval, semantic consistency, and geometry verification, achieving high accuracy on multiple challenging datasets.

Contribution

It presents an integrated coarse-to-fine re-localization pipeline with a novel observation constraints module for improved outlier filtering.

Findings

Enhanced localization accuracy on Aachen Day-Night and InLoc datasets.

Improved initial pose estimation through cascade retrieval and spatial verification.

Effective filtering of outliers using geometry and semantic consistency.

Abstract

Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline is designed as a coarse-to-fine paradigm. In the retrieval part, we cascade the architecture of ResNet101-GeM-ArcFace and employ DBSCAN followed by spatial verification to obtain a better initial coarse pose. We design a module called observation constraints, which combines geometry information and semantic consistency for filtering outliers. Comprehensive experiments are conducted on open datasets, including retrieval on R-Oxford5k and R-Paris6k, semantic segmentation on Cityscapes, localization on Aachen Day-Night and InLoc. By creatively modifying separate modules in the total pipeline, our method achieves many performance improvements on the challenging localization benchmarks.