Name That Part: 3D Part Segmentation and Naming

TL;DR

ALIGN-Parts is a novel 3D part segmentation and naming method that aligns shape parts with textual descriptions using a set matching approach, enabling open-vocabulary, zero-shot, and scalable annotation of 3D objects.

Contribution

We introduce ALIGN-Parts, a set alignment-based approach for 3D part segmentation and naming that integrates geometric, visual, and semantic cues for open-vocabulary matching.

Findings

Supports zero-shot matching to arbitrary descriptions

Creates a unified ontology of 3D parts from multiple datasets

Introduces two new metrics for named 3D part segmentation

Abstract

We address semantic 3D part segmentation: decomposing objects into parts with meaningful names. While datasets exist with part annotations, their definitions are inconsistent across datasets, limiting robust training. Previous methods produce unlabeled decompositions or retrieve single parts without complete shape annotations. We propose ALIGN-Parts, which formulates part naming as a direct set alignment task. Our method decomposes shapes into partlets - implicit 3D part representations - matched to part descriptions via bipartite assignment. We combine geometric cues from 3D part fields, appearance cues from multi-view vision features, and semantic knowledge from language-model-generated affordance descriptions. Text-alignment loss ensures partlets share embedding space with text, enabling a theoretically open-vocabulary matching setup, given sufficient data. Our efficient and novel, one-shot, 3D part segmentation and naming method finds applications in several downstream tasks, including serving as a scalable annotation engine. As our model supports zero-shot matching to arbitrary descriptions and confidence-calibrated predictions for known categories, with human verification, we create a unified ontology that aligns PartNet, 3DCoMPaT++, and Find3D, consisting of 1,794 unique 3D parts. We introduce two novel metrics appropriate for the named 3D part segmentation task. We also show examples from our newly created TexParts dataset.