Perspective from a Higher Dimension: Can 3D Geometric Priors Help Visual Floorplan Localization?

TL;DR

This paper introduces a novel 3D geometric prior approach for visual floorplan localization, leveraging multi-view constraints and scene surface reconstruction, significantly improving accuracy over existing methods without extra annotations.

Contribution

It proposes a new method that injects 3D geometric priors into 2D floorplan localization using self-supervised contrastive learning, addressing visual changes and occlusions.

Findings

Outperforms state-of-the-art methods in localization accuracy

Effectively bridges modal gaps with 3D priors

No additional annotations needed for modeling

Abstract

Since a building's floorplans are easily accessible, consistent over time, and inherently robust to changes in visual appearance, self-localization within the floorplan has attracted researchers' interest. However, since floorplans are minimalist representations of a building's structure, modal and geometric differences between visual perceptions and floorplans pose challenges to this task. While existing methods cleverly utilize 2D geometric features and pose filters to achieve promising performance, they fail to address the localization errors caused by frequent visual changes and view occlusions due to variously shaped 3D objects. To tackle these issues, this paper views the 2D Floorplan Localization (FLoc) problem from a higher dimension by injecting 3D geometric priors into the visual FLoc algorithm. For the 3D geometric prior modeling, we first model geometrically aware view invariance using multi-view constraints, i.e., leveraging imaging geometric principles to provide matching constraints between multiple images that see the same points. Then, we further model the view-scene aligned geometric priors, enhancing the cross-modal geometry-color correspondences by associating the scene's surface reconstruction with the RGB frames of the sequence. Both 3D priors are modeled through self-supervised contrastive learning, thus no additional geometric or semantic annotations are required. These 3D priors summarized in extensive realistic scenes bridge the modal gap while improving localization success without increasing the computational burden on the FLoc algorithm. Sufficient comparative studies demonstrate that our method significantly outperforms state-of-the-art methods and substantially boosts the FLoc accuracy. All data and code will be released after the anonymous review.