Learning to Generate 3D Shapes from a Single Example

TL;DR

This paper introduces a novel deep generative model that creates diverse 3D shapes from a single reference shape using a multi-scale GAN and tri-plane representation, avoiding large datasets.

Contribution

The authors develop a single-example 3D shape generation method employing a multi-scale GAN with a tri-plane hybrid representation, enabling high-quality shape synthesis without external data.

Findings

Generates diverse 3D shapes with variations across scales.

Retains global structure of the reference shape.

Operates efficiently using 2D convolutions on voxel pyramids.

Abstract

Existing generative models for 3D shapes are typically trained on a large 3D dataset, often of a specific object category. In this paper, we investigate the deep generative model that learns from only a single reference 3D shape. Specifically, we present a multi-scale GAN-based model designed to capture the input shape's geometric features across a range of spatial scales. To avoid large memory and computational cost induced by operating on the 3D volume, we build our generator atop the tri-plane hybrid representation, which requires only 2D convolutions. We train our generative model on a voxel pyramid of the reference shape, without the need of any external supervision or manual annotation. Once trained, our model can generate diverse and high-quality 3D shapes possibly of different sizes and aspect ratios. The resulting shapes present variations across different scales, and at the same time retain the global structure of the reference shape. Through extensive evaluation, both qualitative and quantitative, we demonstrate that our model can generate 3D shapes of various types.