WorldGrow: Generating Infinite 3D World

TL;DR

WorldGrow introduces a hierarchical 3D scene generation framework that leverages pre-trained models to produce infinite, coherent, and photorealistic virtual environments, overcoming previous scalability and consistency challenges.

Contribution

It presents a novel hierarchical method combining data curation, scene inpainting, and coarse-to-fine generation for scalable, infinite 3D scene synthesis.

Findings

Achieves state-of-the-art geometry reconstruction on 3D-FRONT dataset.

Supports infinite scene generation with consistent and realistic visuals.

Demonstrates potential for large-scale virtual environment creation.

Abstract

We tackle the challenge of generating the infinitely extendable 3D world -- large, continuous environments with coherent geometry and realistic appearance. Existing methods face key challenges: 2D-lifting approaches suffer from geometric and appearance inconsistencies across views, 3D implicit representations are hard to scale up, and current 3D foundation models are mostly object-centric, limiting their applicability to scene-level generation. Our key insight is leveraging strong generation priors from pre-trained 3D models for structured scene block generation. To this end, we propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis. Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity. Evaluated on the large-scale 3D-FRONT dataset, WorldGrow achieves SOTA performance in geometry reconstruction, while uniquely supporting infinite scene generation with photorealistic and structurally consistent outputs. These results highlight its capability for constructing large-scale virtual environments and potential for building future world models.