Hierarchical Transformers for Unsupervised 3D Shape Abstraction

TL;DR

This paper presents HiT, a hierarchical transformer model that learns unsupervised 3D shape hierarchies across categories, enabling detailed shape segmentation without predefined structures.

Contribution

It introduces a flexible hierarchical transformer with a codebook for unsupervised 3D shape abstraction across diverse categories, surpassing fixed-structure limitations.

Findings

Successfully segments shapes into multiple hierarchy levels

Captures meaningful containment relationships

Works across all 55 ShapeNet categories

Abstract

We introduce HiT, a novel hierarchical neural field representation for 3D shapes that learns general hierarchies in a coarse-to-fine manner across different shape categories in an unsupervised setting. Our key contribution is a hierarchical transformer (HiT), where each level learns parent-child relationships of the tree hierarchy using a compressed codebook. This codebook enables the network to automatically identify common substructures across potentially diverse shape categories. Unlike previous works that constrain the task to a fixed hierarchical structure (e.g., binary), we impose no such restriction, except for limiting the total number of nodes at each tree level. This flexibility allows our method to infer the hierarchical structure directly from data, over multiple shape categories, and representing more general and complex hierarchies than prior approaches. When trained at scale with a reconstruction loss, our model captures meaningful containment relationships between parent and child nodes. We demonstrate its effectiveness through an unsupervised shape segmentation task over all 55 ShapeNet categories, where our method successfully segments shapes into multiple levels of granularity.