ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusion

TL;DR

ShapeShifter is a 3D generative model that efficiently creates detailed shape variations from a single reference, combining multiscale neural architecture with sparse voxel and point sampling for improved detail and flexibility.

Contribution

It introduces a novel multiscale neural architecture combining sparse voxel grids and point sampling for efficient, detailed 3D shape variation generation from a single model.

Findings

Better capture of fine geometric details.

Handles more general surface types than previous methods.

Supports interactive and human-controlled shape variation.

Abstract

This paper proposes ShapeShifter, a new 3D generative model that learns to synthesize shape variations based on a single reference model. While generative methods for 3D objects have recently attracted much attention, current techniques often lack geometric details and/or require long training times and large resources. Our approach remedies these issues by combining sparse voxel grids and point, normal, and color sampling within a multiscale neural architecture that can be trained efficiently and in parallel. We show that our resulting variations better capture the fine details of their original input and can handle more general types of surfaces than previous SDF-based methods. Moreover, we offer interactive generation of 3D shape variants, allowing more human control in the design loop if needed.