Dense Semantic 3D Map Based Long-Term Visual Localization with Hybrid Features

TL;DR

This paper introduces a long-term visual localization approach that combines hybrid handcrafted and learned features with dense semantic 3D maps to improve robustness against appearance variations.

Contribution

It proposes a novel method integrating hybrid features and dense semantic maps for enhanced long-term visual localization accuracy.

Findings

Outperforms state-of-the-art methods on long-term localization benchmarks.

Utilizes semantic consistency scores to improve 2D-3D matching reliability.

Demonstrates robustness under varying environmental conditions.

Abstract

Visual localization plays an important role in many applications. However, due to the large appearance variations such as season and illumination changes, as well as weather and day-night variations, it's still a big challenge for robust long-term visual localization algorithms. In this paper, we present a novel visual localization method using hybrid handcrafted and learned features with dense semantic 3D map. Hybrid features help us to make full use of their strengths in different imaging conditions, and the dense semantic map provide us reliable and complete geometric and semantic information for constructing sufficient 2D-3D matching pairs with semantic consistency scores. In our pipeline, we retrieve and score each candidate database image through the semantic consistency between the dense model and the query image. Then the semantic consistency score is used as a soft constraint in the weighted RANSAC-based PnP pose solver. Experimental results on long-term visual localization benchmarks demonstrate the effectiveness of our method compared with state-of-the-arts.