SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generation

TL;DR

SPA-VAE is an unsupervised generative model for 3D point clouds that leverages parts-based self-similarity to improve accuracy, consistency, and efficiency in modeling objects with repeating parts.

Contribution

It introduces a novel variational Bayesian framework with shared parts assignment and self-similarity modeling for unsupervised 3D point cloud generation.

Findings

Outperforms existing methods on ShapeNet datasets

Effectively models self-similar parts within objects

Produces accurate and consistent generated point clouds

Abstract

This paper addresses the problem of unsupervised parts-aware point cloud generation with learned parts-based self-similarity. Our SPA-VAE infers a set of latent canonical candidate shapes for any given object, along with a set of rigid body transformations for each such candidate shape to one or more locations within the assembled object. In this way, noisy samples on the surface of, say, each leg of a table, are effectively combined to estimate a single leg prototype. When parts-based self-similarity exists in the raw data, sharing data among parts in this way confers numerous advantages: modeling accuracy, appropriately self-similar generative outputs, precise in-filling of occlusions, and model parsimony. SPA-VAE is trained end-to-end using a variational Bayesian approach which uses the Gumbel-softmax trick for the shared part assignments, along with various novel losses to provide appropriate inductive biases. Quantitative and qualitative analyses on ShapeNet demonstrate the advantage of SPA-VAE.