WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generation

TL;DR

WorldFlow3D introduces a flow-based method for unbounded 3D scene generation, enabling high-quality, controllable, and cross-domain scene synthesis with rapid convergence.

Contribution

It presents a novel latent-free flow approach that models 3D data distributions for unbounded scene generation, surpassing existing methods in quality and speed.

Findings

Generates causal and accurate 3D structures.

Guides complex scene and texture generation effectively.

Demonstrates high-quality results on real and synthetic data.

Abstract

Unbounded 3D world generation is emerging as a foundational task for scene modeling in computer vision, graphics, and robotics. In this work, we present WorldFlow3D, a novel method capable of generating unbounded 3D worlds. Building upon a foundational property of flow matching - namely, defining a path of transport between two data distributions - we model 3D generation more generally as a problem of flowing through 3D data distributions, not limited to conditional denoising. We find that our latent-free flow approach generates causal and accurate 3D structure, and can use this as an intermediate distribution to guide the generation of more complex structure and high-quality texture - all while converging more rapidly than existing methods. We enable controllability over generated scenes with vectorized scene layout conditions for geometric structure control and visual texture control through scene attributes. We confirm the effectiveness of WorldFlow3D on both real outdoor driving scenes and synthetic indoor scenes, validating cross-domain generalizability and high-quality generation on real data distributions. We confirm favorable scene generation fidelity over approaches in all tested settings for unbounded scene generation. For more, see https://light.princeton.edu/worldflow3d.