3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks

TL;DR

3D-PRNN is a generative recurrent neural network that creates structured 3D shape representations from limited sensor data by synthesizing primitive-based models, capturing symmetry and structural coherence.

Contribution

The paper introduces a novel primitive-based generative model for 3D shapes that encodes symmetry and coherence, trained on a large dataset generated via Gaussian Fields.

Findings

Outperforms nearest-neighbor shape retrieval methods.

Comparable to voxel-based models with fewer parameters.

Effectively captures symmetry and structural features.

Abstract

The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3D-PRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxel-based generative models while using a significantly reduced parameter space.