DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising

TL;DR

DreamPartGen is a novel framework that generates semantically grounded, part-aware 3D objects from text by jointly modeling geometry, appearance, and inter-part relations, achieving state-of-the-art results.

Contribution

It introduces Duplex Part Latents and Relational Semantic Latents for joint geometric and semantic modeling in text-to-3D generation.

Findings

Achieves state-of-the-art geometric fidelity.

Demonstrates superior text-shape alignment.

Produces coherent and interpretable 3D models.

Abstract

Understanding and generating 3D objects as compositions of meaningful parts is fundamental to human perception and reasoning. However, most text-to-3D methods overlook the semantic and functional structure of parts. While recent part-aware approaches introduce decomposition, they remain largely geometry-focused, lacking semantic grounding and failing to model how parts align with textual descriptions or their inter-part relations. We propose DreamPartGen, a framework for semantically grounded, part-aware text-to-3D generation. DreamPartGen introduces Duplex Part Latents (DPLs) that jointly model each part's geometry and appearance, and Relational Semantic Latents (RSLs) that capture inter-part dependencies derived from language. A synchronized co-denoising process enforces mutual geometric and semantic consistency, enabling coherent, interpretable, and text-aligned 3D synthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity and text-shape alignment.