Weakly Supervised LiDAR Semantic Segmentation via Scatter Image Annotation

TL;DR

This paper introduces a novel weakly supervised LiDAR segmentation approach using scatter image annotation, combining efficient annotation with a specialized network to achieve near full supervision performance with minimal labeled data.

Contribution

It proposes ScatterNet, a new network with multimodal fusion and perception consistency loss, and introduces scatter image annotation for rapid dense labeling of LiDAR data.

Findings

Achieves over 95% of fully-supervised performance with less than 0.02% labeled points.

Uses only 5% of the labeled points compared to state-of-the-art weakly supervised methods.

Demonstrates effectiveness on nuScenes and SemanticKITTI datasets.

Abstract

Weakly supervised LiDAR semantic segmentation has made significant strides with limited labeled data. However, most existing methods focus on the network training under weak supervision, while efficient annotation strategies remain largely unexplored. To tackle this gap, we implement LiDAR semantic segmentation using scatter image annotation, effectively integrating an efficient annotation strategy with network training. Specifically, we propose employing scatter images to annotate LiDAR point clouds, combining a pre-trained optical flow estimation network with a foundation image segmentation model to rapidly propagate manual annotations into dense labels for both images and point clouds. Moreover, we propose ScatterNet, a network that includes three pivotal strategies to reduce the performance gap caused by such annotations. Firstly, it utilizes dense semantic labels as supervision for the image branch, alleviating the modality imbalance between point clouds and images. Secondly, an intermediate fusion branch is proposed to obtain multimodal texture and structural features. Lastly, a perception consistency loss is introduced to determine which information needs to be fused and which needs to be discarded during the fusion process. Extensive experiments on the nuScenes and SemanticKITTI datasets have demonstrated that our method requires less than 0.02% of the labeled points to achieve over 95% of the performance of fully-supervised methods. Notably, our labeled points are only 5% of those used in the most advanced weakly supervised methods.