SeDAR - Semantic Detection and Ranging: Humans can localise without LiDAR, can robots?

TL;DR

This paper introduces SeDAR, a semantic-based localization method that enables robots to localize using only semantic labels from RGB images and floorplans, mimicking human localization strategies and achieving state-of-the-art results without relying on depth data.

Contribution

SeDAR presents a novel semantic-driven localization approach that does not require depth measurements, contrasting with traditional scan-matching and vision-based methods.

Findings

Achieves localization accuracy comparable to state-of-the-art methods without depth data.

Utilizes semantic labels from RGB images and floorplans for effective global localization.

Demonstrates robustness even with inaccurate or incomplete floorplans.

Abstract

How does a person work out their location using a floorplan? It is probably safe to say that we do not explicitly measure depths to every visible surface and try to match them against different pose estimates in the floorplan. And yet, this is exactly how most robotic scan-matching algorithms operate. Similarly, we do not extrude the 2D geometry present in the floorplan into 3D and try to align it to the real-world. And yet, this is how most vision-based approaches localise. Humans do the exact opposite. Instead of depth, we use high level semantic cues. Instead of extruding the floorplan up into the third dimension, we collapse the 3D world into a 2D representation. Evidence of this is that many of the floorplans we use in everyday life are not accurate, opting instead for high levels of discriminative landmarks. In this work, we use this insight to present a global localisation approach that relies solely on the semantic labels present in the floorplan and extracted from RGB images. While our approach is able to use range measurements if available, we demonstrate that they are unnecessary as we can achieve results comparable to state-of-the-art without them.