Deformable 3D Shape Diffusion Model

TL;DR

This paper introduces a deformable 3D shape diffusion model that enhances 3D shape manipulation by incorporating a differential deformation kernel within a probabilistic diffusion framework, enabling advanced applications like facial animation.

Contribution

It presents a novel deformable diffusion approach with a differential deformation kernel for versatile 3D shape manipulation, including point clouds, meshes, and animations.

Findings

State-of-the-art in point cloud generation

Competitive results in mesh deformation

Effective for facial animation and shape editing

Abstract

The Gaussian diffusion model, initially designed for image generation, has recently been adapted for 3D point cloud generation. However, these adaptations have not fully considered the intrinsic geometric characteristics of 3D shapes, thereby constraining the diffusion model's potential for 3D shape manipulation. To address this limitation, we introduce a novel deformable 3D shape diffusion model that facilitates comprehensive 3D shape manipulation, including point cloud generation, mesh deformation, and facial animation. Our approach innovatively incorporates a differential deformation kernel, which deconstructs the generation of geometric structures into successive non-rigid deformation stages. By leveraging a probabilistic diffusion model to simulate this step-by-step process, our method provides a versatile and efficient solution for a wide range of applications, spanning from graphics rendering to facial expression animation. Empirical evidence highlights the effectiveness of our approach, demonstrating state-of-the-art performance in point cloud generation and competitive results in mesh deformation. Additionally, extensive visual demonstrations reveal the significant potential of our approach for practical applications. Our method presents a unique pathway for advancing 3D shape manipulation and unlocking new opportunities in the realm of virtual reality.