Hierarchical Image-Guided 3D Point Cloud Segmentation in Industrial Scenes via Multi-View Bayesian Fusion

TL;DR

This paper introduces a hierarchical, image-guided 3D segmentation framework for industrial scenes that refines segmentation from instance to part level using multi-view Bayesian fusion, effectively handling occlusion and complexity.

Contribution

The proposed method uniquely combines multi-view image rendering, SAM and YOLO-World prompts, and Bayesian fusion for accurate, annotation-efficient 3D segmentation in complex industrial environments.

Findings

Achieves high per-class mIoU scores on real-world factory data.

Effectively handles occlusion and structural complexity.

Demonstrates robustness and generalization on public datasets.

Abstract

Reliable 3D segmentation is critical for understanding complex scenes with dense layouts and multi-scale objects, as commonly seen in industrial environments. In such scenarios, heavy occlusion weakens geometric boundaries between objects, and large differences in object scale will cause end-to-end models fail to capture both coarse and fine details accurately. Existing 3D point-based methods require costly annotations, while image-guided methods often suffer from semantic inconsistencies across views. To address these challenges, we propose a hierarchical image-guided 3D segmentation framework that progressively refines segmentation from instance-level to part-level. Instance segmentation involves rendering a top-view image and projecting SAM-generated masks prompted by YOLO-World back onto the 3D point cloud. Part-level segmentation is subsequently performed by rendering multi-view images of each instance obtained from the previous stage and applying the same 2D segmentation and back-projection process at each view, followed by Bayesian updating fusion to ensure semantic consistency across views. Experiments on real-world factory data demonstrate that our method effectively handles occlusion and structural complexity, achieving consistently high per-class mIoU scores. Additional evaluations on public dataset confirm the generalization ability of our framework, highlighting its robustness, annotation efficiency, and adaptability to diverse 3D environments.