MeshDiffusion: Score-based Generative 3D Mesh Modeling

TL;DR

MeshDiffusion introduces a novel diffusion-based generative model for creating detailed 3D meshes using deformable tetrahedral grids, improving over prior methods in realism and geometric detail.

Contribution

The paper presents a new diffusion model for mesh generation that directly models deformable tetrahedral grids, avoiding post-processing and enhancing geometric detail.

Findings

Effective generation of realistic 3D meshes

Produces detailed and fine-grained geometric features

Outperforms previous methods in mesh quality

Abstract

We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parametrization. We demonstrate the effectiveness of our model on multiple generative tasks.