PanopticNDT: Efficient and Robust Panoptic Mapping

TL;DR

PanopticNDT introduces an efficient real-time 3D panoptic mapping method for mobile robots, combining high-detail scene understanding with robustness and suitability for indoor environments.

Contribution

It presents a novel occupancy NDT-based approach that achieves high-resolution, real-time panoptic mapping on mobile robots, outperforming existing methods.

Findings

Higher level of detail in panoptic representation

Real-time mapping capability on mobile robots

Effective in indoor domestic environments

Abstract

As the application scenarios of mobile robots are getting more complex and challenging, scene understanding becomes increasingly crucial. A mobile robot that is supposed to operate autonomously in indoor environments must have precise knowledge about what objects are present, where they are, what their spatial extent is, and how they can be reached; i.e., information about free space is also crucial. Panoptic mapping is a powerful instrument providing such information. However, building 3D panoptic maps with high spatial resolution is challenging on mobile robots, given their limited computing capabilities. In this paper, we propose PanopticNDT - an efficient and robust panoptic mapping approach based on occupancy normal distribution transform (NDT) mapping. We evaluate our approach on the publicly available datasets Hypersim and ScanNetV2. The results reveal that our approach can represent panoptic information at a higher level of detail than other state-of-the-art approaches while enabling real-time panoptic mapping on mobile robots. Finally, we prove the real-world applicability of PanopticNDT with qualitative results in a domestic application.