SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering

TL;DR

SMAC-Seg introduces a real-time LiDAR panoptic segmentation method that employs sparse multi-directional attention clustering and a novel centroid-aware repel loss, achieving state-of-the-art results on major datasets.

Contribution

The paper proposes a novel sparse multi-directional attention clustering method with a centroid-aware repel loss for efficient LiDAR panoptic segmentation.

Findings

Achieves state-of-the-art performance on SemanticKITTI and nuScenes datasets.

Operates in real-time without complex proposal networks.

Effectively differentiates object clusters using the new loss function.

Abstract

Panoptic segmentation aims to address semantic and instance segmentation simultaneously in a unified framework. However, an efficient solution of panoptic segmentation in applications like autonomous driving is still an open research problem. In this work, we propose a novel LiDAR-based panoptic system, called SMAC-Seg. We present a learnable sparse multi-directional attention clustering to segment multi-scale foreground instances. SMAC-Seg is a real-time clustering-based approach, which removes the complex proposal network to segment instances. Most existing clustering-based methods use the difference of the predicted and ground truth center offset as the only loss to supervise the instance centroid regression. However, this loss function only considers the centroid of the current object, but its relative position with respect to the neighbouring objects is not considered when learning to cluster. Thus, we propose to use a novel centroid-aware repel loss as an additional term to effectively supervise the network to differentiate each object cluster with its neighbours. Our experimental results show that SMAC-Seg achieves state-of-the-art performance among all real-time deployable networks on both large-scale public SemanticKITTI and nuScenes panoptic segmentation datasets.