DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation

TL;DR

DAE-Net is an unsupervised 3D shape co-segmentation method that learns deformable part templates to segment diverse shapes into meaningful, consistent parts, using a novel autoencoder architecture with deformation networks.

Contribution

It introduces a deformable autoencoder with a per-part deformation network for fine-grained, unsupervised 3D shape co-segmentation across diverse datasets.

Findings

Achieves superior segmentation accuracy on ShapeNet Part dataset

Produces consistent, meaningful parts across diverse shapes

Outperforms prior unsupervised co-segmentation methods

Abstract

We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection. To accommodate structural variations in the collection, our network composes each shape by a selected subset of template parts which are affine-transformed. To maximize the expressive power of the part templates, we introduce a per-part deformation network to enable the modeling of diverse parts with substantial geometry variations, while imposing constraints on the deformation capacity to ensure fidelity to the originally represented parts. We also propose a training scheme to effectively overcome local minima. Architecturally, our network is a branched autoencoder, with a CNN encoder taking a voxel shape as input and producing per-part transformation matrices, latent codes, and part existence scores, and the decoder outputting point occupancies to define the reconstruction loss. Our network, coined DAE-Net for Deforming Auto-Encoder, can achieve unsupervised 3D shape co-segmentation that yields fine-grained, compact, and meaningful parts that are consistent across diverse shapes. We conduct extensive experiments on the ShapeNet Part dataset, DFAUST, and an animal subset of Objaverse to show superior performance over prior methods. Code and data are available at https://github.com/czq142857/DAE-Net.