Canonical Capsules: Self-Supervised Capsules in Canonical Pose
Weiwei Sun, Andrea Tagliasacchi, Boyang Deng, Sara Sabour, Soroosh, Yazdani, Geoffrey Hinton, Kwang Moo Yi

TL;DR
This paper introduces a self-supervised capsule network architecture for 3D point clouds that learns semantically meaningful, canonical object representations without labeled data, improving reconstruction and classification tasks.
Contribution
The authors develop a novel self-supervised capsule model that learns canonical object decompositions and object-centric reasoning without requiring labels or aligned datasets.
Findings
Outperforms state-of-the-art in 3D reconstruction
Achieves superior canonicalization of objects
Enables unsupervised classification of 3D point clouds
Abstract
We propose a self-supervised capsule architecture for 3D point clouds. We compute capsule decompositions of objects through permutation-equivariant attention, and self-supervise the process by training with pairs of randomly rotated objects. Our key idea is to aggregate the attention masks into semantic keypoints, and use these to supervise a decomposition that satisfies the capsule invariance/equivariance properties. This not only enables the training of a semantically consistent decomposition, but also allows us to learn a canonicalization operation that enables object-centric reasoning. To train our neural network we require neither classification labels nor manually-aligned training datasets. Yet, by learning an object-centric representation in a self-supervised manner, our method outperforms the state-of-the-art on 3D point cloud reconstruction, canonicalization, and unsupervised…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
Topics3D Shape Modeling and Analysis · Human Pose and Action Recognition · Image Processing and 3D Reconstruction
