TL;DR
Lyra 2.0 introduces a scalable framework for generating long, consistent 3D worlds by addressing spatial forgetting and temporal drifting in video-based scene creation.
Contribution
It proposes a novel method that maintains per-frame 3D geometry and trains with self-augmented histories to improve long-horizon, 3D-consistent video generation.
Findings
Enables longer, more consistent 3D scene trajectories.
Improves scene appearance and geometry fidelity over extended sequences.
Facilitates reliable 3D scene reconstruction from generated videos.
Abstract
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via feed-forward reconstruction techniques. This generative reconstruction approach combines the visual fidelity and creative capacity of video models with 3D outputs ready for real-time rendering and simulation. Scaling to large, complex environments requires 3D-consistent video generation over long camera trajectories with large viewpoint changes and location revisits, a setting where current video models degrade quickly. Existing methods for long-horizon generation are fundamentally limited by two forms of degradation: spatial forgetting and temporal drifting. As exploration proceeds, previously observed regions fall outside the model's temporal context, forcing the model to hallucinate structures when…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
