BuildingBlock: A Hybrid Approach for Structured Building Generation

TL;DR

BuildingBlock introduces a hybrid pipeline combining generative models, procedural content generation, and large language models to produce diverse, structured, and hierarchically coherent 3D buildings for virtual applications.

Contribution

It presents a novel two-phase approach integrating Transformer-based diffusion models and LLMs for globally consistent layout and hierarchical building design.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Generates diverse and hierarchically structured buildings.

Enables scalable and intuitive architectural workflows.

Abstract

Three-dimensional building generation is vital for applications in gaming, virtual reality, and digital twins, yet current methods face challenges in producing diverse, structured, and hierarchically coherent buildings. We propose BuildingBlock, a hybrid approach that integrates generative models, procedural content generation (PCG), and large language models (LLMs) to address these limitations. Specifically, our method introduces a two-phase pipeline: the Layout Generation Phase (LGP) and the Building Construction Phase (BCP). LGP reframes box-based layout generation as a point-cloud generation task, utilizing a newly constructed architectural dataset and a Transformer-based diffusion model to create globally consistent layouts. With LLMs, these layouts are extended into rule-based hierarchical designs, seamlessly incorporating component styles and spatial structures. The BCP leverages these layouts to guide PCG, enabling local-customizable, high-quality structured building generation. Experimental results demonstrate BuildingBlock's effectiveness in generating diverse and hierarchically structured buildings, achieving state-of-the-art results on multiple benchmarks, and paving the way for scalable and intuitive architectural workflows.