PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusion

TL;DR

PartDiffuser is a semi-autoregressive diffusion framework that generates detailed 3D meshes by combining global topology control with high-frequency local feature reconstruction, outperforming existing methods.

Contribution

It introduces a novel part-wise diffusion approach with a part-aware cross-attention mechanism for improved 3D mesh generation.

Findings

Outperforms SOTA models in 3D mesh quality

Effectively decouples global and local generation tasks

Produces meshes with rich, high-fidelity details

Abstract

Existing autoregressive (AR) methods for generating artist-designed meshes struggle to balance global structural consistency with high-fidelity local details, and are susceptible to error accumulation. To address this, we propose PartDiffuser, a novel semi-autoregressive diffusion framework for point-cloud-to-mesh generation. The method first performs semantic segmentation on the mesh and then operates in a "part-wise" manner: it employs autoregression between parts to ensure global topology, while utilizing a parallel discrete diffusion process within each semantic part to precisely reconstruct high-frequency geometric features. PartDiffuser is based on the DiT architecture and introduces a part-aware cross-attention mechanism, using point clouds as hierarchical geometric conditioning to dynamically control the generation process, thereby effectively decoupling the global and local generation tasks. Experiments demonstrate that this method significantly outperforms state-of-the-art (SOTA) models in generating 3D meshes with rich detail, exhibiting exceptional detail representation suitable for real-world applications.