Beyond the Label Itself: Latent Labels Enhance Semi-supervised Point Cloud Panoptic Segmentation

TL;DR

This paper introduces a semi-supervised point cloud panoptic segmentation method that leverages latent labels from LiDAR and image data, employing novel augmentation and learning modules to improve performance on benchmark datasets.

Contribution

It proposes two new latent label-based techniques, Cylinder-Mix augmentation and IPSL module, to enhance semi-supervised segmentation performance.

Findings

Outperforms state-of-the-art LaserMix on SemanticKITTI and nuScenes

Demonstrates robustness of the IPSL module

Effectively utilizes latent labels from multi-modal data

Abstract

As the exorbitant expense of labeling autopilot datasets and the growing trend of utilizing unlabeled data, semi-supervised segmentation on point clouds becomes increasingly imperative. Intuitively, finding out more ``unspoken words'' (i.e., latent instance information) beyond the label itself should be helpful to improve performance. In this paper, we discover two types of latent labels behind the displayed label embedded in LiDAR and image data. First, in the LiDAR Branch, we propose a novel augmentation, Cylinder-Mix, which is able to augment more yet reliable samples for training. Second, in the Image Branch, we propose the Instance Position-scale Learning (IPSL) Module to learn and fuse the information of instance position and scale, which is from a 2D pre-trained detector and a type of latent label obtained from 3D to 2D projection. Finally, the two latent labels are embedded into the multi-modal panoptic segmentation network. The ablation of the IPSL module demonstrates its robust adaptability, and the experiments evaluated on SemanticKITTI and nuScenes demonstrate that our model outperforms the state-of-the-art method, LaserMix.