CoCo4D: Comprehensive and Complex 4D Scene Generation
Junwei Zhou, Xueting Li, Lu Qi, Ming-Hsuan Yang

TL;DR
CoCo4D is a novel framework that generates detailed, multi-view consistent 4D scenes from text prompts by modeling foreground motion and background evolution separately, ensuring realistic and immersive dynamic scene synthesis.
Contribution
The paper introduces CoCo4D, a new method that divides 4D scene generation into foreground and background modeling guided by motion, enabling more realistic and coherent dynamic scene synthesis from text and images.
Findings
Achieves comparable or superior 4D scene generation quality to existing methods.
Effectively models articulated foreground motion and background changes.
Demonstrates efficiency and realism in dynamic scene synthesis.
Abstract
Existing 4D synthesis methods primarily focus on object-level generation or dynamic scene synthesis with limited novel views, restricting their ability to generate multi-view consistent and immersive dynamic 4D scenes. To address these constraints, we propose a framework (dubbed as CoCo4D) for generating detailed dynamic 4D scenes from text prompts, with the option to include images. Our method leverages the crucial observation that articulated motion typically characterizes foreground objects, whereas background alterations are less pronounced. Consequently, CoCo4D divides 4D scene synthesis into two responsibilities: modeling the dynamic foreground and creating the evolving background, both directed by a reference motion sequence. Given a text prompt and an optional reference image, CoCo4D first generates an initial motion sequence utilizing video diffusion models. This motion…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topics3D Modeling in Geospatial Applications · 3D Shape Modeling and Analysis · Image Processing and 3D Reconstruction
MethodsDiffusion · Focus
