Volumetric Semantically Consistent 3D Panoptic Mapping

TL;DR

This paper presents an online 3D semantic mapping method that integrates confidence measures and graph optimization to produce accurate, consistent 3D maps for autonomous agents, outperforming existing approaches.

Contribution

It introduces a novel semantic 3D mapping algorithm combining confidence integration and graph optimization, improving accuracy over state-of-the-art methods.

Findings

Achieves superior accuracy on large-scale datasets.

Highlights the impact of trajectory estimation on mapping performance.

Improves semantic and instance consistency in 3D maps.

Abstract

We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating comprehensive, accurate, and efficient semantic 3D maps suitable for autonomous agents in unstructured environments. The proposed approach is based on a Voxel-TSDF representation used in recent algorithms. It introduces novel ways of integrating semantic prediction confidence during mapping, producing semantic and instance-consistent 3D regions. Further improvements are achieved by graph optimization-based semantic labeling and instance refinement. The proposed method achieves accuracy superior to the state of the art on public large-scale datasets, improving on a number of widely used metrics. We also highlight a downfall in the evaluation of recent studies: using the ground truth trajectory as input instead of a SLAM-estimated one substantially affects the accuracy, creating a large gap between the reported results and the actual performance on real-world data.