Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmap

TL;DR

Panoptic-PHNet is a novel LiDAR-based framework that achieves real-time, high-precision panoptic segmentation by introducing a clustering pseudo heatmap, a knn-transformer module, and a fused backbone design, outperforming existing methods.

Contribution

The paper presents a new clustering pseudo heatmap paradigm, a knn-transformer for foreground interaction modeling, and a fused backbone for detailed and global feature utilization, advancing LiDAR panoptic segmentation.

Findings

Surpasses state-of-the-art on SemanticKITTI and nuScenes datasets.

Achieves real-time processing speeds.

Secures 1st place on SemanticKITTI leaderboard.

Abstract

As a rising task, panoptic segmentation is faced with challenges in both semantic segmentation and instance segmentation. However, in terms of speed and accuracy, existing LiDAR methods in the field are still limited. In this paper, we propose a fast and high-performance LiDAR-based framework, referred to as Panoptic-PHNet, with three attractive aspects: 1) We introduce a clustering pseudo heatmap as a new paradigm, which, followed by a center grouping module, yields instance centers for efficient clustering without object-level learning tasks. 2) A knn-transformer module is proposed to model the interaction among foreground points for accurate offset regression. 3) For backbone design, we fuse the fine-grained voxel features and the 2D Bird's Eye View (BEV) features with different receptive fields to utilize both detailed and global information. Extensive experiments on both SemanticKITTI dataset and nuScenes dataset show that our Panoptic-PHNet surpasses state-of-the-art methods by remarkable margins with a real-time speed. We achieve the 1st place on the public leaderboard of SemanticKITTI and leading performance on the recently released leaderboard of nuScenes.