Learning from Spatio-temporal Correlation for Semi-Supervised LiDAR Semantic Segmentation

TL;DR

This paper introduces a semi-supervised LiDAR segmentation method that leverages spatio-temporal priors and a dual-branch structure to improve pseudo-label quality and handle data imbalance, achieving state-of-the-art results with limited labeled data.

Contribution

It proposes a proximity-based pseudo-label estimation method utilizing temporal overlap and a dual-branch architecture to enhance semi-supervised LiDAR segmentation performance.

Findings

Achieves state-of-the-art results on SemanticKITTI and nuScenes datasets.

Performs competitively with fully-supervised methods using only 5% labeled data.

Surpasses previous methods at 20% labeled data on nuScenes.

Abstract

We address the challenges of the semi-supervised LiDAR segmentation (SSLS) problem, particularly in low-budget scenarios. The two main issues in low-budget SSLS are the poor-quality pseudo-labels for unlabeled data, and the performance drops due to the significant imbalance between ground-truth and pseudo-labels. This imbalance leads to a vicious training cycle. To overcome these challenges, we leverage the spatio-temporal prior by recognizing the substantial overlap between temporally adjacent LiDAR scans. We propose a proximity-based label estimation, which generates highly accurate pseudo-labels for unlabeled data by utilizing semantic consistency with adjacent labeled data. Additionally, we enhance this method by progressively expanding the pseudo-labels from the nearest unlabeled scans, which helps significantly reduce errors linked to dynamic classes. Additionally, we employ a dual-branch structure to mitigate performance degradation caused by data imbalance. Experimental results demonstrate remarkable performance in low-budget settings (i.e., <= 5%) and meaningful improvements in normal budget settings (i.e., 5 - 50%). Finally, our method has achieved new state-of-the-art results on SemanticKITTI and nuScenes in semi-supervised LiDAR segmentation. With only 5% labeled data, it offers competitive results against fully-supervised counterparts. Moreover, it surpasses the performance of the previous state-of-the-art at 100% labeled data (75.2%) using only 20% of labeled data (76.0%) on nuScenes. The code is available on https://github.com/halbielee/PLE.