Part2Object: Hierarchical Unsupervised 3D Instance Segmentation

TL;DR

Part2Object introduces a hierarchical clustering approach guided by 3D objectness priors from 2D frames, enabling improved unsupervised 3D instance segmentation and hierarchical object part segmentation.

Contribution

It presents a novel hierarchical clustering method with object guidance and a new model Hi-Mask3D for hierarchical 3D segmentation, advancing unsupervised 3D instance segmentation.

Findings

Outperforms state-of-the-art in unsupervised segmentation

Supports data-efficient fine-tuning and cross-dataset generalization

Provides hierarchical 3D object part and instance segmentation

Abstract

Unsupervised 3D instance segmentation aims to segment objects from a 3D point cloud without any annotations. Existing methods face the challenge of either too loose or too tight clustering, leading to under-segmentation or over-segmentation. To address this issue, we propose Part2Object, hierarchical clustering with object guidance. Part2Object employs multi-layer clustering from points to object parts and objects, allowing objects to manifest at any layer. Additionally, it extracts and utilizes 3D objectness priors from temporally consecutive 2D RGB frames to guide the clustering process. Moreover, we propose Hi-Mask3D to support hierarchical 3D object part and instance segmentation. By training Hi-Mask3D on the objects and object parts extracted from Part2Object, we achieve consistent and superior performance compared to state-of-the-art models in various settings, including unsupervised instance segmentation, data-efficient fine-tuning, and cross-dataset generalization. Code is release at https://github.com/ChengShiest/Part2Object