Learning Fine-Grained Segmentation of 3D Shapes without Part Labels

TL;DR

This paper introduces a deep clustering approach for fine-grained 3D shape segmentation that does not require part labels, enabling detailed segmentation of complex models with high accuracy.

Contribution

It proposes a novel deep clustering framework with a low rank loss and divide-and-conquer strategy for label-free fine-grained 3D shape segmentation.

Findings

Achieves state-of-the-art results on fine-grained segmentation benchmarks.

Effectively handles densely sampled point clouds.

Demonstrates robustness without requiring semantic part labels.

Abstract

Learning-based 3D shape segmentation is usually formulated as a semantic labeling problem, assuming that all parts of training shapes are annotated with a given set of tags. This assumption, however, is impractical for learning fine-grained segmentation. Although most off-the-shelf CAD models are, by construction, composed of fine-grained parts, they usually miss semantic tags and labeling those fine-grained parts is extremely tedious. We approach the problem with deep clustering, where the key idea is to learn part priors from a shape dataset with fine-grained segmentation but no part labels. Given point sampled 3D shapes, we model the clustering priors of points with a similarity matrix and achieve part segmentation through minimizing a novel low rank loss. To handle highly densely sampled point sets, we adopt a divide-and-conquer strategy. We partition the large point set into a number of blocks. Each block is segmented using a deep-clustering-based part prior network trained in a category-agnostic manner. We then train a graph convolution network to merge the segments of all blocks to form the final segmentation result. Our method is evaluated with a challenging benchmark of fine-grained segmentation, showing state-of-the-art performance.