X-Dyna: Expressive Dynamic Human Image Animation
Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai,, Chenxu Zhang, Guoxian Song, Chao Wang, Yichun Shi, Zeyuan Chen, Shijie Zhou,, Linjie Luo, Gordon Wetzstein, Mohammad Soleymani

TL;DR
X-Dyna is a diffusion-based framework that animates a single human image with realistic, context-aware facial and body movements, improving lifelikeness over previous pose-based methods.
Contribution
It introduces the Dynamics-Adapter and local control modules, enabling detailed, identity-disentangled facial expressions and scene dynamics in zero-shot human image animation.
Findings
Outperforms state-of-the-art methods in realism and expressiveness
Generates fluid, context-aware animations with detailed dynamics
Achieves high-quality results in diverse scenes and motions
Abstract
We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · 3D Shape Modeling and Analysis · Computer Graphics and Visualization Techniques
MethodsDiffusion
