SegMap: Segment-based mapping and localization using data-driven descriptors

TL;DR

SegMap introduces a segment-based 3D point cloud mapping and localization method that uses data-driven descriptors for improved accuracy, invariance, and real-time processing in complex environments.

Contribution

It presents a novel segment-based map representation using learned descriptors that enhance localization and mapping in unstructured, dynamic environments.

Findings

Superior segment retrieval capabilities compared to handcrafted descriptors.

Achieved higher localization accuracy and 6% increase in recall.

Reduced odometry drift by up to 50%.

Abstract

Precisely estimating a robot's pose in a prior, global map is a fundamental capability for mobile robotics, e.g. autonomous driving or exploration in disaster zones. This task, however, remains challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural changes, and facilitates real-time processing of large-scale 3D data. SegMap exploits a single compact data-driven descriptor for performing multiple tasks: global localization, 3D dense map reconstruction, and semantic information extraction. The performance of SegMap is evaluated in multiple urban driving and search and rescue experiments. We show that the learned SegMap descriptor has superior segment retrieval capabilities, compared to state-of-the-art handcrafted descriptors. In consequence, we achieve a higher localization accuracy and a 6% increase in recall over state-of-the-art. These segment-based localizations allow us to reduce the open-loop odometry drift by up to 50%. SegMap is open-source available along with easy to run demonstrations.