Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion

TL;DR

Assembler introduces a scalable 3D part assembly framework using diffusion models and anchor point representations, enabling high-quality assembly of diverse, real-world objects from parts and images.

Contribution

It presents a novel generative approach with shape-centric anchor point representation and a large-scale dataset for scalable, generalizable 3D part assembly.

Findings

State-of-the-art performance on PartNet dataset

First to demonstrate assembly of complex real-world objects

Enables high-resolution, editable 3D object generation from images

Abstract

We present Assembler, a scalable and generalizable framework for 3D part assembly that reconstructs complete objects from input part meshes and a reference image. Unlike prior approaches that mostly rely on deterministic part pose prediction and category-specific training, Assembler is designed to handle diverse, in-the-wild objects with varying part counts, geometries, and structures. It addresses the core challenges of scaling to general 3D part assembly through innovations in task formulation, representation, and data. First, Assembler casts part assembly as a generative problem and employs diffusion models to sample plausible configurations, effectively capturing ambiguities arising from symmetry, repeated parts, and multiple valid assemblies. Second, we introduce a novel shape-centric representation based on sparse anchor point clouds, enabling scalable generation in Euclidean space rather than SE(3) pose prediction. Third, we construct a large-scale dataset of over 320K diverse part-object assemblies using a synthesis and filtering pipeline built on existing 3D shape repositories. Assembler achieves state-of-the-art performance on PartNet and is the first to demonstrate high-quality assembly for complex, real-world objects. Based on Assembler, we further introduce an interesting part-aware 3D modeling system that generates high-resolution, editable objects from images, demonstrating potential for interactive and compositional design. Project page: https://assembler3d.github.io