PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes

TL;DR

PQ-NET is a deep neural network that models 3D shapes as sequences of parts, enabling shape reconstruction, interpolation, and generation through a sequential part assembly process.

Contribution

It introduces a novel Seq2Seq autoencoder for 3D shape representation and generation based on part sequences, capturing both structure and geometry.

Findings

Effective shape autoencoding and reconstruction.

Ability to generate novel 3D shapes with meaningful parts.

Successful interpolation and single-view reconstruction.

Abstract

We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly. The input to our network is a 3D shape segmented into parts, where each part is first encoded into a feature representation using a part autoencoder. The core component of PQ-NET is a sequence-to-sequence or Seq2Seq autoencoder which encodes a sequence of part features into a latent vector of fixed size, and the decoder reconstructs the 3D shape, one part at a time, resulting in a sequential assembly. The latent space formed by the Seq2Seq encoder encodes both part structure and fine part geometry. The decoder can be adapted to perform several generative tasks including shape autoencoding, interpolation, novel shape generation, and single-view 3D reconstruction, where the generated shapes are all composed of meaningful parts.