X-Dyna: Expressive Dynamic Human Image Animation

TL;DR

X-Dyna is a diffusion-based framework that animates a single human image with realistic, context-aware facial and body movements, improving lifelikeness over previous pose-based methods.

Contribution

It introduces the Dynamics-Adapter and local control modules, enabling detailed, identity-disentangled facial expressions and scene dynamics in zero-shot human image animation.

Findings

Outperforms state-of-the-art methods in realism and expressiveness

Generates fluid, context-aware animations with detailed dynamics

Achieves high-quality results in diverse scenes and motions

Abstract

We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at https://github.com/bytedance/X-Dyna.