GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis

TL;DR

GEM3D introduces a topology-aware generative model for 3D shapes that uses neural skeleton representations and diffusion models to produce accurate, diverse, and complex 3D surface reconstructions.

Contribution

The paper presents a novel skeleton-based neural implicit framework with diffusion modeling for topologically accurate 3D shape synthesis and reconstruction.

Findings

Outperforms previous methods in surface fidelity and diversity.

Effectively reconstructs high-genus, complex shapes from challenging datasets.

Demonstrates superior results in shape synthesis and point cloud reconstruction.

Abstract

We introduce GEM3D -- a new deep, topology-aware generative model of 3D shapes. The key ingredient of our method is a neural skeleton-based representation encoding information on both shape topology and geometry. Through a denoising diffusion probabilistic model, our method first generates skeleton-based representations following the Medial Axis Transform (MAT), then generates surfaces through a skeleton-driven neural implicit formulation. The neural implicit takes into account the topological and geometric information stored in the generated skeleton representations to yield surfaces that are more topologically and geometrically accurate compared to previous neural field formulations. We discuss applications of our method in shape synthesis and point cloud reconstruction tasks, and evaluate our method both qualitatively and quantitatively. We demonstrate significantly more faithful surface reconstruction and diverse shape generation results compared to the state-of-the-art, also involving challenging scenarios of reconstructing and synthesizing structurally complex, high-genus shape surfaces from Thingi10K and ShapeNet.