Multi-dataset synergistic in supervised learning to pre-label structural components in point clouds from shell construction scenes

TL;DR

This paper presents a method using pre-trained transformer models and transfer learning to efficiently segment complex structural components in point clouds from shell construction scenes, reducing annotation effort.

Contribution

It adapts existing datasets and transformer architectures for the challenging task of structural component segmentation in construction site point clouds, demonstrating effective transfer learning.

Findings

Pre-trained transformers achieve effective segmentation with minimal fine-tuning.

Transfer learning enhances performance on new construction site data.

Method reduces annotation effort for large-scale construction datasets.

Abstract

The significant effort required to annotate data for new training datasets hinders computer vision research and machine learning in the construction industry. This work explores adapting standard datasets and the latest transformer model architectures for point cloud semantic segmentation in the context of shell construction sites. Unlike common approaches focused on object segmentation of building interiors and furniture, this study addressed the challenges of segmenting complex structural components in Architecture, Engineering, and Construction (AEC). We establish a baseline through supervised training and a custom validation dataset, evaluate the cross-domain inference with large-scale indoor datasets, and utilize transfer learning to maximize segmentation performance with minimal new data. The findings indicate that with minimal fine-tuning, pre-trained transformer architectures offer an effective strategy for building component segmentation. Our results are promising for automating the annotation of new, previously unseen data when creating larger training resources and for the segmentation of frequently recurring objects.