CRAG: Can 3D Generative Models Help 3D Assembly?

TL;DR

CRAG introduces a novel joint approach to 3D assembly that combines shape generation and pose estimation, leading to improved handling of incomplete and complex objects.

Contribution

This work reformulates 3D assembly as a combined generation and assembly problem, enabling synthesis of missing geometry and better pose prediction.

Findings

Achieves state-of-the-art results on diverse 3D objects

Handles missing parts effectively

Outperforms prior pose estimation methods

Abstract

Most existing 3D assembly methods treat the problem as pure pose estimation, rearranging observed parts via rigid transformations. In contrast, human assembly naturally couples structural reasoning with holistic shape inference. Inspired by this intuition, we reformulate 3D assembly as a joint problem of assembly and generation. We show that these two processes are mutually reinforcing: assembly provides part-level structural priors for generation, while generation injects holistic shape context that resolves ambiguities in assembly. Unlike prior methods that cannot synthesize missing geometry, we propose CRAG, which simultaneously generates plausible complete shapes and predicts poses for input parts. Extensive experiments demonstrate state-of-the-art performance across in-the-wild objects with diverse geometries, varying part counts, and missing pieces. Our code and models will be released.