Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

TL;DR

This paper introduces panoptic multi-TSDFs, a novel multi-resolution volumetric mapping method that maintains semantic consistency and adapts resolution dynamically for long-term dynamic scene understanding in robotics.

Contribution

The paper presents a new panoptic multi-TSDFs representation that integrates high-level semantic information for efficient, accurate, and adaptive 3D mapping in changing environments.

Findings

Efficient online map construction and maintenance.

High accuracy in dynamic scene reconstruction.

Robust operation with real depth sensors.

Abstract

For robotic interaction in environments shared with other agents, access to volumetric and semantic maps of the scene is crucial. However, such environments are inevitably subject to long-term changes, which the map needs to account for. We thus propose panoptic multi-TSDFs as a novel representation for multi-resolution volumetric mapping in changing environments. By leveraging high-level information for 3D reconstruction, our proposed system allocates high resolution only where needed. Through reasoning on the object level, semantic consistency over time is achieved. This enables our method to maintain up-to-date reconstructions with high accuracy while improving coverage by incorporating previous data. We show in thorough experimental evaluation that our map can be efficiently constructed, maintained, and queried during online operation, and that the presented approach can operate robustly on real depth sensors using non-optimized panoptic segmentation as input.