Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation

TL;DR

This paper introduces a real-time, scalable semantic mapping method that combines geometric segmentation with CNN-based refinement, achieving high accuracy and efficiency in 3D map building.

Contribution

It presents a novel incremental semantic mapping approach that runs at over 30Hz, integrating SLAM, segmentation, and recognition with high efficiency and accuracy.

Findings

Operates at over 30Hz for real-time processing

Achieves high accuracy with CNN-based segmentation refinement

Validated on NYUv2 dataset showing competitive performance

Abstract

We propose an efficient and scalable method for incrementally building a dense, semantically annotated 3D map in real-time. The proposed method assigns class probabilities to each region, not each element (e.g., surfel and voxel), of the 3D map which is built up through a robust SLAM framework and incrementally segmented with a geometric-based segmentation method. Differently from all other approaches, our method has a capability of running at over 30Hz while performing all processing components, including SLAM, segmentation, 2D recognition, and updating class probabilities of each segmentation label at every incoming frame, thanks to the high efficiency that characterizes the computationally intensive stages of our framework. By utilizing a specifically designed CNN to improve the frame-wise segmentation result, we can also achieve high accuracy. We validate our method on the NYUv2 dataset by comparing with the state of the art in terms of accuracy and computational efficiency, and by means of an analysis in terms of time and space complexity.