SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation

TL;DR

SP-GAN is an unsupervised 3D shape generator that uses sphere-guided priors to produce diverse, detailed point clouds with controllable part-based manipulation capabilities, without requiring part annotations.

Contribution

It introduces a novel sphere-guided generative model that disentangles global shape modeling from local detail adjustment for improved 3D shape synthesis and manipulation.

Findings

Produces high-quality, diverse 3D shapes with fine details

Enables part-aware shape editing and interpolation

Outperforms state-of-the-art models in quality and controllability

Abstract

We present SP-GAN, a new unsupervised sphere-guided generative model for direct synthesis of 3D shapes in the form of point clouds. Compared with existing models, SP-GAN is able to synthesize diverse and high-quality shapes with fine details and promote controllability for part-aware shape generation and manipulation, yet trainable without any parts annotations. In SP-GAN, we incorporate a global prior (uniform points on a sphere) to spatially guide the generative process and attach a local prior (a random latent code) to each sphere point to provide local details. The key insight in our design is to disentangle the complex 3D shape generation task into a global shape modeling and a local structure adjustment, to ease the learning process and enhance the shape generation quality. Also, our model forms an implicit dense correspondence between the sphere points and points in every generated shape, enabling various forms of structure-aware shape manipulations such as part editing, part-wise shape interpolation, and multi-shape part composition, etc., beyond the existing generative models. Experimental results, which include both visual and quantitative evaluations, demonstrate that our model is able to synthesize diverse point clouds with fine details and less noise, as compared with the state-of-the-art models.