Generating Physically Stable and Buildable Brick Structures from Text
Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu, Jun-Yan Zhu

TL;DR
BrickGPT is a novel AI approach that generates physically stable, buildable brick structures from text prompts, combining large-scale datasets, physics-aware inference, and texturing for diverse, aesthetic designs suitable for human and robotic assembly.
Contribution
We introduce BrickGPT, the first large language model trained to generate physically stable brick structures from text, with a new dataset and physics-aware inference techniques.
Findings
Produces stable, diverse brick structures aligned with text prompts
Enables manual and robotic assembly of generated designs
Provides a large dataset of over 47,000 structures with captions
Abstract
We introduce BrickGPT, the first approach for generating physically stable interconnecting brick assembly models from text prompts. To achieve this, we construct a large-scale, physically stable dataset of brick structures, along with their associated captions, and train an autoregressive large language model to predict the next brick to add via next-token prediction. To improve the stability of the resulting designs, we employ an efficient validity check and physics-aware rollback during autoregressive inference, which prunes infeasible token predictions using physics laws and assembly constraints. Our experiments show that BrickGPT produces stable, diverse, and aesthetically pleasing brick structures that align closely with the input text prompts. We also develop a text-based brick texturing method to generate colored and textured designs. We show that our designs can be assembled…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
