From One to More: Contextual Part Latents for 3D Generation

TL;DR

This paper introduces CoPart, a part-aware 3D diffusion framework that decomposes objects into parts for improved multi-part generation, control, and coherence, supported by a new large-scale dataset Partverse.

Contribution

It proposes a novel part-aware diffusion approach with explicit part modeling and a mutual guidance strategy, advancing 3D generation and editing capabilities.

Findings

Enhanced part-level editing and scene composition.

Improved articulation and geometric coherence.

Superior controllability over 3D object generation.

Abstract

Recent advances in 3D generation have transitioned from multi-view 2D rendering approaches to 3D-native latent diffusion frameworks that exploit geometric priors in ground truth data. Despite progress, three key limitations persist: (1) Single-latent representations fail to capture complex multi-part geometries, causing detail degradation; (2) Holistic latent coding neglects part independence and interrelationships critical for compositional design; (3) Global conditioning mechanisms lack fine-grained controllability. Inspired by human 3D design workflows, we propose CoPart - a part-aware diffusion framework that decomposes 3D objects into contextual part latents for coherent multi-part generation. This paradigm offers three advantages: i) Reduces encoding complexity through part decomposition; ii) Enables explicit part relationship modeling; iii) Supports part-level conditioning. We further develop a mutual guidance strategy to fine-tune pre-trained diffusion models for joint part latent denoising, ensuring both geometric coherence and foundation model priors. To enable large-scale training, we construct Partverse - a novel 3D part dataset derived from Objaverse through automated mesh segmentation and human-verified annotations. Extensive experiments demonstrate CoPart's superior capabilities in part-level editing, articulated object generation, and scene composition with unprecedented controllability.