4D Panoptic Segmentation as Invariant and Equivariant Field Prediction

TL;DR

This paper introduces rotation-equivariant neural networks for 4D panoptic segmentation in autonomous driving, improving accuracy and efficiency by leveraging symmetry in LiDAR data, and achieves state-of-the-art results on SemanticKITTI.

Contribution

It develops invariant and equivariant field prediction methods for 4D panoptic segmentation, enhancing generalization and robustness in autonomous driving scenarios.

Findings

Achieves higher accuracy than non-equivariant models

Reduces computational costs

Sets new state-of-the-art on SemanticKITTI leaderboard

Abstract

In this paper, we develop rotation-equivariant neural networks for 4D panoptic segmentation. 4D panoptic segmentation is a benchmark task for autonomous driving that requires recognizing semantic classes and object instances on the road based on LiDAR scans, as well as assigning temporally consistent IDs to instances across time. We observe that the driving scenario is symmetric to rotations on the ground plane. Therefore, rotation-equivariance could provide better generalization and more robust feature learning. Specifically, we review the object instance clustering strategies and restate the centerness-based approach and the offset-based approach as the prediction of invariant scalar fields and equivariant vector fields. Other sub-tasks are also unified from this perspective, and different invariant and equivariant layers are designed to facilitate their predictions. Through evaluation on the standard 4D panoptic segmentation benchmark of SemanticKITTI, we show that our equivariant models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Moreover, our method sets the new state-of-the-art performance and achieves 1st place on the SemanticKITTI 4D Panoptic Segmentation leaderboard.