3DSES: an indoor Lidar point cloud segmentation dataset with real and pseudo-labels from a 3D model

TL;DR

This paper introduces 3DSES, a new indoor TLS point cloud dataset with real and pseudo-labels, enabling improved semantic segmentation for BIM and robotics applications, and demonstrates the effectiveness of automated labeling using 3D CAD models.

Contribution

The paper presents a novel dataset with a unique double annotation format and an automated labeling method using 3D CAD models, facilitating research in indoor point cloud segmentation.

Findings

Pseudo-labels achieve over 95% accuracy in model-to-cloud alignment.

Leveraging pseudo-labels and Lidar intensity improves segmentation accuracy.

Existing models struggle with BIM-relevant objects, highlighting dataset challenges.

Abstract

Semantic segmentation of indoor point clouds has found various applications in the creation of digital twins for robotics, navigation and building information modeling (BIM). However, most existing datasets of labeled indoor point clouds have been acquired by photogrammetry. In contrast, Terrestrial Laser Scanning (TLS) can acquire dense sub-centimeter point clouds and has become the standard for surveyors. We present 3DSES (3D Segmentation of ESGT point clouds), a new dataset of indoor dense TLS colorized point clouds covering 427 m 2 of an engineering school. 3DSES has a unique double annotation format: semantic labels annotated at the point level alongside a full 3D CAD model of the building. We introduce a model-to-cloud algorithm for automated labeling of indoor point clouds using an existing 3D CAD model. 3DSES has 3 variants of various semantic and geometrical complexities. We show that our model-to-cloud alignment can produce pseudo-labels on our point clouds with a \> 95% accuracy, allowing us to train deep models with significant time savings compared to manual labeling. First baselines on 3DSES show the difficulties encountered by existing models when segmenting objects relevant to BIM, such as light and safety utilities. We show that segmentation accuracy can be improved by leveraging pseudo-labels and Lidar intensity, an information rarely considered in current datasets. Code and data will be open sourced.