SegMap: 3D Segment Mapping using Data-Driven Descriptors

TL;DR

SegMap introduces a data-driven 3D segment mapping approach that enhances localization, map reconstruction, and semantic understanding in robotics, demonstrated through urban and rescue experiments with improved accuracy over existing methods.

Contribution

The paper presents a novel data-driven descriptor for 3D segments that supports localization, dense map reconstruction, and semantic extraction, advancing beyond current single-task methods.

Findings

28.3% improvement in ROC area over state-of-the-art eigenvalue features

Comparable reconstruction quality to specialized models

Effective in urban driving and search and rescue scenarios

Abstract

When performing localization and mapping, working at the level of structure can be advantageous in terms of robustness to environmental changes and differences in illumination. This paper presents SegMap: a map representation solution to the localization and mapping problem based on the extraction of segments in 3D point clouds. In addition to facilitating the computationally intensive task of processing 3D point clouds, working at the level of segments addresses the data compression requirements of real-time single- and multi-robot systems. While current methods extract descriptors for the single task of localization, SegMap leverages a data-driven descriptor in order to extract meaningful features that can also be used for reconstructing a dense 3D map of the environment and for extracting semantic information. This is particularly interesting for navigation tasks and for providing visual feedback to end-users such as robot operators, for example in search and rescue scenarios. These capabilities are demonstrated in multiple urban driving and search and rescue experiments. Our method leads to an increase of area under the ROC curve of 28.3% over current state of the art using eigenvalue based features. We also obtain very similar reconstruction capabilities to a model specifically trained for this task. The SegMap implementation will be made available open-source along with easy to run demonstrations at www.github.com/ethz-asl/segmap. A video demonstration is available at https://youtu.be/CMk4w4eRobg.