WorldExplorer: Towards Generating Fully Navigable 3D Scenes

TL;DR

WorldExplorer is a novel method that generates fully navigable 3D scenes from text by iteratively creating videos along trajectories, ensuring visual consistency and enabling realistic exploration.

Contribution

It introduces an autoregressive video trajectory generation approach with scene memory and collision detection for high-quality, explorable 3D scene synthesis from text prompts.

Findings

Produces stable, high-quality 3D scenes under large camera motions

Enables realistic unrestricted exploration of generated environments

Fuses multi-view videos into unified 3D representations

Abstract

Generating 3D worlds from text is a highly anticipated goal in computer vision. Existing works are limited by the degree of exploration they allow inside of a scene, i.e., produce streched-out and noisy artifacts when moving beyond central or panoramic perspectives. To this end, we propose WorldExplorer, a novel method based on autoregressive video trajectory generation, which builds fully navigable 3D scenes with consistent visual quality across a wide range of viewpoints. We initialize our scenes by creating multi-view consistent images corresponding to a 360 degree panorama. Then, we expand it by leveraging video diffusion models in an iterative scene generation pipeline. Concretely, we generate multiple videos along short, pre-defined trajectories, that explore the scene in depth, including motion around objects. Our novel scene memory conditions each video on the most relevant prior views, while a collision-detection mechanism prevents degenerate results, like moving into objects. Finally, we fuse all generated views into a unified 3D representation via 3D Gaussian Splatting optimization. Compared to prior approaches, WorldExplorer produces high-quality scenes that remain stable under large camera motion, enabling for the first time realistic and unrestricted exploration. We believe this marks a significant step toward generating immersive and truly explorable virtual 3D environments.