U3DS$^3$: Unsupervised 3D Semantic Scene Segmentation

TL;DR

U3DS$^3$ introduces a fully unsupervised method for 3D scene segmentation that works on both indoor and outdoor point clouds without pre-training, using geometric features and clustering to achieve state-of-the-art results.

Contribution

This work presents the first fully unsupervised approach for holistic 3D scene segmentation applicable to various environments, eliminating the need for annotated data and pre-training.

Findings

Achieves state-of-the-art results on ScanNet and SemanticKITTI datasets.

Demonstrates effective unsupervised segmentation across indoor and outdoor scenes.

Provides competitive results on the S3DIS dataset.

Abstract

Contemporary point cloud segmentation approaches largely rely on richly annotated 3D training data. However, it is both time-consuming and challenging to obtain consistently accurate annotations for such 3D scene data. Moreover, there is still a lack of investigation into fully unsupervised scene segmentation for point clouds, especially for holistic 3D scenes. This paper presents U3DS$^3$, as a step towards completely unsupervised point cloud segmentation for any holistic 3D scenes. To achieve this, U3DS$^3$ leverages a generalized unsupervised segmentation method for both object and background across both indoor and outdoor static 3D point clouds with no requirement for model pre-training, by leveraging only the inherent information of the point cloud to achieve full 3D scene segmentation. The initial step of our proposed approach involves generating superpoints based on the geometric characteristics of each scene. Subsequently, it undergoes a learning process through a spatial clustering-based methodology, followed by iterative training using pseudo-labels generated in accordance with the cluster centroids. Moreover, by leveraging the invariance and equivariance of the volumetric representations, we apply the geometric transformation on voxelized features to provide two sets of descriptors for robust representation learning. Finally, our evaluation provides state-of-the-art results on the ScanNet and SemanticKITTI, and competitive results on the S3DIS, benchmark datasets.