PaintScene4D: Consistent 4D Scene Generation from Text Prompts
Vinayak Gupta, Yunze Man, Yu-Xiong Wang

TL;DR
PaintScene4D introduces a training-free framework that generates photorealistic, consistent 4D scenes from text prompts by leveraging video generative models and strategic rendering techniques, enabling flexible camera control and realistic scene synthesis.
Contribution
It presents a novel, training-free approach for 4D scene generation from text, utilizing video models and multi-view rendering for improved realism and control.
Findings
Produces photorealistic 4D scenes from text prompts.
Ensures spatial and temporal consistency across viewpoints.
Enables flexible camera trajectories during rendering.
Abstract
Recent advances in diffusion models have revolutionized 2D and 3D content creation, yet generating photorealistic dynamic 4D scenes remains a significant challenge. Existing dynamic 4D generation methods typically rely on distilling knowledge from pre-trained 3D generative models, often fine-tuned on synthetic object datasets. Consequently, the resulting scenes tend to be object-centric and lack photorealism. While text-to-video models can generate more realistic scenes with motion, they often struggle with spatial understanding and provide limited control over camera viewpoints during rendering. To address these limitations, we present PaintScene4D, a novel text-to-4D scene generation framework that departs from conventional multi-view generative models in favor of a streamlined architecture that harnesses video generative models trained on diverse real-world datasets. Our method first…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · Handwritten Text Recognition Techniques · Multimodal Machine Learning Applications
MethodsDiffusion · Inpainting
