AnimateScene: Camera-controllable Animation in Any Scene

TL;DR

AnimateScene is a comprehensive framework that enables realistic, camera-controlled 4D human animations integrated seamlessly into diverse 3D scenes, addressing placement, style, and motion coherence challenges.

Contribution

It introduces novel modules for automatic human placement, style alignment without training, and joint post-reconstruction with camera trajectories, advancing scene-aware animation synthesis.

Findings

Produces high-detail, coherent animations across scenes

Automatically prevents human-scene interpenetration

Achieves smooth camera-controlled motion videos

Abstract

Recent advances in 3D scene reconstruction and 4D human animation have broadened adoption, but integrating the two remains difficult. Key challenges include placing humans at plausible locations and scales without interpenetration, aligning lighting and style between humans and background, and handling dynamic camera trajectories. We introduce AnimateScene, a unified framework addressing these issues. First, an accurate placement module automatically determines realistic 3D positions and prevents collisions during motion. Second, we propose a training-free style alignment method that adapts the 4D human to match background illumination and appearance, achieving coherent composites. Finally, we design a joint post-reconstruction approach that incorporates camera trajectories for smooth, visually engaging motion videos. Experiments across diverse scenes and actions demonstrate that AnimateScene generates dynamic results with high geometric detail and strong spatiotemporal coherence. The project page is available at https://whynothaha.github.io/AnimateScene/AnimateScene.html.