StackGen: Generating Stable Structures from Silhouettes via Diffusion

TL;DR

StackGen is a diffusion-based model that generates diverse, stable object configurations matching a silhouette, enabling robots to reason about stable structures without detailed geometric models.

Contribution

The paper introduces StackGen, a novel diffusion model that predicts stable object arrangements from silhouettes, bridging intuitive physics and robotic assembly.

Findings

Successfully generates stable configurations matching target silhouettes.

Demonstrates real-world robotic assembly of generated structures.

Effective in simulated and real environments.

Abstract

Humans naturally obtain intuition about the interactions between and the stability of rigid objects by observing and interacting with the world. It is this intuition that governs the way in which we regularly configure objects in our environment, allowing us to build complex structures from simple, everyday objects. Robotic agents, on the other hand, traditionally require an explicit model of the world that includes the detailed geometry of each object and an analytical model of the environment dynamics, which are difficult to scale and preclude generalization. Instead, robots would benefit from an awareness of intuitive physics that enables them to similarly reason over the stable interaction of objects in their environment. Towards that goal, we propose StackGen, a diffusion model that generates diverse stable configurations of building blocks matching a target silhouette. To demonstrate the capability of the method, we evaluate it in a simulated environment and deploy it in the real setting using a robotic arm to assemble structures generated by the model.