WonderJourney: Going from Anywhere to Everywhere

TL;DR

WonderJourney is a modular framework that creates coherent, diverse 3D scene sequences from user inputs, enabling immersive virtual journeys through text and image-guided scene generation and verification.

Contribution

It introduces a novel, integrated approach combining LLMs, text-driven point cloud generation, and VLM verification for continuous 3D scene creation from arbitrary starting points.

Findings

Produces diverse, coherent 3D scene sequences

Works from various user inputs like text and images

Generates visually compelling virtual journeys

Abstract

We introduce WonderJourney, a modularized framework for perpetual 3D scene generation. Unlike prior work on view generation that focuses on a single type of scenes, we start at any user-provided location (by a text description or an image) and generate a journey through a long sequence of diverse yet coherently connected 3D scenes. We leverage an LLM to generate textual descriptions of the scenes in this journey, a text-driven point cloud generation pipeline to make a compelling and coherent sequence of 3D scenes, and a large VLM to verify the generated scenes. We show compelling, diverse visual results across various scene types and styles, forming imaginary "wonderjourneys". Project website: https://kovenyu.com/WonderJourney/