MRGAN: Multi-Rooted 3D Shape Generation with Unsupervised Part Disentanglement

TL;DR

MRGAN is a novel 3D shape generation method that produces part-disentangled point clouds without supervision, enabling controllable shape modeling through multiple learnable branches and innovative loss functions.

Contribution

Introduces a multi-rooted adversarial network with a root-mixing strategy and novel loss terms for unsupervised part disentanglement in 3D shape generation.

Findings

Achieves effective part disentanglement in 3D point clouds.

Enables controllable shape manipulation such as part mixing.

Outperforms baseline methods in qualitative and quantitative evaluations.

Abstract

We present MRGAN, a multi-rooted adversarial network which generates part-disentangled 3D point-cloud shapes without part-based shape supervision. The network fuses multiple branches of tree-structured graph convolution layers which produce point clouds, with learnable constant inputs at the tree roots. Each branch learns to grow a different shape part, offering control over the shape generation at the part level. Our network encourages disentangled generation of semantic parts via two key ingredients: a root-mixing training strategy which helps decorrelate the different branches to facilitate disentanglement, and a set of loss terms designed with part disentanglement and shape semantics in mind. Of these, a novel convexity loss incentivizes the generation of parts that are more convex, as semantic parts tend to be. In addition, a root-dropping loss further ensures that each root seeds a single part, preventing the degeneration or over-growth of the point-producing branches. We evaluate the performance of our network on a number of 3D shape classes, and offer qualitative and quantitative comparisons to previous works and baseline approaches. We demonstrate the controllability offered by our part-disentangled generation through two applications for shape modeling: part mixing and individual part variation, without receiving segmented shapes as input.