4D Panoptic LiDAR Segmentation

TL;DR

This paper introduces 4D panoptic LiDAR segmentation, which assigns semantic labels and consistent instance IDs to sequences of 3D LiDAR points over time, improving dynamic scene understanding for autonomous systems.

Contribution

It presents a novel approach modeling object instances as probability distributions in 4D space and introduces a new evaluation metric separating semantic and instance association.

Findings

Effective point-to-instance association without explicit temporal data matching

Parallel processing of multiple point clouds enhances temporal consistency

New metric improves evaluation of temporal LiDAR segmentation

Abstract

Temporal semantic scene understanding is critical for self-driving cars or robots operating in dynamic environments. In this paper, we propose 4D panoptic LiDAR segmentation to assign a semantic class and a temporally-consistent instance ID to a sequence of 3D points. To this end, we present an approach and a point-centric evaluation metric. Our approach determines a semantic class for every point while modeling object instances as probability distributions in the 4D spatio-temporal domain. We process multiple point clouds in parallel and resolve point-to-instance associations, effectively alleviating the need for explicit temporal data association. Inspired by recent advances in benchmarking of multi-object tracking, we propose to adopt a new evaluation metric that separates the semantic and point-to-instance association aspects of the task. With this work, we aim at paving the road for future developments of temporal LiDAR panoptic perception.