3D Shape Generation with Grid-based Implicit Functions

TL;DR

This paper introduces a grid-based implicit function approach for 3D shape generation, enabling more expressive and spatially controllable models that outperform existing methods on standard evaluation metrics.

Contribution

It proposes a novel GAN training method on grid cells with local latent vectors, allowing for better expressiveness and spatial control in 3D shape generation.

Findings

Outperforms current state-of-the-art on established evaluation metrics.

Enables spatial control of shape generation via bounding boxes.

Identifies limitations of existing evaluation measures and proposes a robust alternative.

Abstract

Previous approaches to generate shapes in a 3D setting train a GAN on the latent space of an autoencoder (AE). Even though this produces convincing results, it has two major shortcomings. As the GAN is limited to reproduce the dataset the AE was trained on, we cannot reuse a trained AE for novel data. Furthermore, it is difficult to add spatial supervision into the generation process, as the AE only gives us a global representation. To remedy these issues, we propose to train the GAN on grids (i.e. each cell covers a part of a shape). In this representation each cell is equipped with a latent vector provided by an AE. This localized representation enables more expressiveness (since the cell-based latent vectors can be combined in novel ways) as well as spatial control of the generation process (e.g. via bounding boxes). Our method outperforms the current state of the art on all established evaluation measures, proposed for quantitatively evaluating the generative capabilities of GANs. We show limitations of these measures and propose the adaptation of a robust criterion from statistical analysis as an alternative.