Supercharging Floorplan Localization with Semantic Rays

TL;DR

This paper presents a semantic-aware floorplan localization method that jointly estimates depth and semantic rays, significantly improving accuracy and efficiency over existing techniques by leveraging rich semantic information.

Contribution

The work introduces a novel framework that combines depth and semantic rays for more accurate floorplan localization, incorporating a coarse-to-fine probability volume refinement process.

Findings

Outperforms state-of-the-art methods in recall metrics

Effectively incorporates additional metadata like room labels

Achieves significant accuracy and efficiency improvements

Abstract

Floorplans provide a compact representation of the building's structure, revealing not only layout information but also detailed semantics such as the locations of windows and doors. However, contemporary floorplan localization techniques mostly focus on matching depth-based structural cues, ignoring the rich semantics communicated within floorplans. In this work, we introduce a semantic-aware localization framework that jointly estimates depth and semantic rays, consolidating over both for predicting a structural-semantic probability volume. Our probability volume is constructed in a coarse-to-fine manner: We first sample a small set of rays to obtain an initial low-resolution probability volume. We then refine these probabilities by performing a denser sampling only in high-probability regions and process the refined values for predicting a 2D location and orientation angle. We conduct an evaluation on two standard floorplan localization benchmarks. Our experiments demonstrate that our approach substantially outperforms state-of-the-art methods, achieving significant improvements in recall metrics compared to prior works. Moreover, we show that our framework can easily incorporate additional metadata such as room labels, enabling additional gains in both accuracy and efficiency.