HierOctFusion: Multi-scale Octree-based 3D Shape Generation via Part-Whole-Hierarchy Message Passing

TL;DR

HierOctFusion is a novel multi-scale octree diffusion model that incorporates part hierarchies and semantic information to generate detailed, sparse 3D shapes more efficiently than previous holistic approaches.

Contribution

It introduces a part-aware hierarchical octree diffusion framework with cross-attention for semantic part integration, and creates a new annotated dataset for training and evaluation.

Findings

Achieves higher shape quality than prior methods.

Demonstrates improved efficiency in 3D shape generation.

Effectively propagates semantic features across hierarchical levels.

Abstract

3D content generation remains a fundamental yet challenging task due to the inherent structural complexity of 3D data. While recent octree-based diffusion models offer a promising balance between efficiency and quality through hierarchical generation, they often overlook two key insights: 1) existing methods typically model 3D objects as holistic entities, ignoring their semantic part hierarchies and limiting generalization; and 2) holistic high-resolution modeling is computationally expensive, whereas real-world objects are inherently sparse and hierarchical, making them well-suited for layered generation. Motivated by these observations, we propose HierOctFusion, a part-aware multi-scale octree diffusion model that enhances hierarchical feature interaction for generating fine-grained and sparse object structures. Furthermore, we introduce a cross-attention conditioning mechanism that injects part-level information into the generation process, enabling semantic features to propagate effectively across hierarchical levels from parts to the whole. Additionally, we construct a 3D dataset with part category annotations using a pre-trained segmentation model to facilitate training and evaluation. Experiments demonstrate that HierOctFusion achieves superior shape quality and efficiency compared to prior methods.