Learning to Group and Label Fine-Grained Shape Components

TL;DR

This paper introduces a learning-based method for grouping and labeling fine-grained shape components in 3D models, leveraging hierarchical grouping, context-aware features, and CRFs to improve semantic segmentation accuracy.

Contribution

It proposes a novel hierarchical grouping and labeling approach that considers modeling components as mid-level hypotheses, along with a new benchmark for component-wise labeling.

Findings

Achieves robust labeling on public 3D shape repositories.

Introduces the first benchmark for component-wise labeling.

Demonstrates significant improvement over existing methods.

Abstract

A majority of stock 3D models in modern shape repositories are assembled with many fine-grained components. The main cause of such data form is the component-wise modeling process widely practiced by human modelers. These modeling components thus inherently reflect some function-based shape decomposition the artist had in mind during modeling. On the other hand, modeling components represent an over-segmentation since a functional part is usually modeled as a multi-component assembly. Based on these observations, we advocate that labeled segmentation of stock 3D models should not overlook the modeling components and propose a learning solution to grouping and labeling of the fine-grained components. However, directly characterizing the shape of individual components for the purpose of labeling is unreliable, since they can be arbitrarily tiny and semantically meaningless. We propose to generate part hypotheses from the components based on a hierarchical grouping strategy, and perform labeling on those part groups instead of directly on the components. Part hypotheses are mid-level elements which are more probable to carry semantic information. A multiscale 3D convolutional neural network is trained to extract context-aware features for the hypotheses. To accomplish a labeled segmentation of the whole shape, we formulate higher-order conditional random fields (CRFs) to infer an optimal label assignment for all components. Extensive experiments demonstrate that our method achieves significantly robust labeling results on raw 3D models from public shape repositories. Our work also contributes the first benchmark for component-wise labeling.