4D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregation

TL;DR

This paper introduces 4D-StOP, a novel approach for 4D LiDAR panoptic segmentation that uses spatio-temporal proposals and geometric aggregation, significantly outperforming existing methods on the SemanticKITTI dataset.

Contribution

The paper presents a new spatio-temporal proposal generation and aggregation method for 4D LiDAR segmentation, surpassing state-of-the-art accuracy.

Findings

Achieves 63.9 LSTQ on SemanticKITTI, a +7% improvement.

Uses voting-based proposals and geometric features for better segmentation.

Outperforms Gaussian distribution-based models in 4D scene understanding.

Abstract

In this work, we present a new paradigm, called 4D-StOP, to tackle the task of 4D Panoptic LiDAR Segmentation. 4D-StOP first generates spatio-temporal proposals using voting-based center predictions, where each point in the 4D volume votes for a corresponding center. These tracklet proposals are further aggregated using learned geometric features. The tracklet aggregation method effectively generates a video-level 4D scene representation over the entire space-time volume. This is in contrast to existing end-to-end trainable state-of-the-art approaches which use spatio-temporal embeddings that are represented by Gaussian probability distributions. Our voting-based tracklet generation method followed by geometric feature-based aggregation generates significantly improved panoptic LiDAR segmentation quality when compared to modeling the entire 4D volume using Gaussian probability distributions. 4D-StOP achieves a new state-of-the-art when applied to the SemanticKITTI test dataset with a score of 63.9 LSTQ, which is a large (+7%) improvement compared to current best-performing end-to-end trainable methods. The code and pre-trained models are available at: https://github.com/LarsKreuzberg/4D-StOP.