High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotation

TL;DR

This paper introduces a novel framework for indoor point cloud segmentation using scene-level annotations, combining multi-modal features and semantic consistency to generate high-quality pseudo-labels and improve segmentation accuracy.

Contribution

It proposes a new pseudo-label generation method leveraging cross-modal guidance and region-point semantic consistency for scene-level annotated point cloud segmentation.

Findings

Significant performance improvements on ScanNet v2 and S3DIS datasets.

Effective pseudo-label refinement during training.

Validation of individual component contributions through ablation studies.

Abstract

This paper investigates indoor point cloud semantic segmentation under scene-level annotation, which is less explored compared to methods relying on sparse point-level labels. In the absence of precise point-level labels, current methods first generate point-level pseudo-labels, which are then used to train segmentation models. However, generating accurate pseudo-labels for each point solely based on scene-level annotations poses a considerable challenge, substantially affecting segmentation performance. Consequently, to enhance accuracy, this paper proposes a high-quality pseudo-label generation framework by exploring contemporary multi-modal information and region-point semantic consistency. Specifically, with a cross-modal feature guidance module, our method utilizes 2D-3D correspondences to align point cloud features with corresponding 2D image pixels, thereby assisting point cloud feature learning. To further alleviate the challenge presented by the scene-level annotation, we introduce a region-point semantic consistency module. It produces regional semantics through a region-voting strategy derived from point-level semantics, which are subsequently employed to guide the point-level semantic predictions. Leveraging the aforementioned modules, our method can rectify inaccurate point-level semantic predictions during training and obtain high-quality pseudo-labels. Significant improvements over previous works on ScanNet v2 and S3DIS datasets under scene-level annotation can demonstrate the effectiveness. Additionally, comprehensive ablation studies validate the contributions of our approach's individual components. The code is available at https://github.com/LHDuan/WSegPC .