SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers

TL;DR

SkyReels-A1 is a novel framework that improves portrait video animation by enhancing identity preservation, facial expression accuracy, and temporal stability using advanced diffusion transformers and multi-stage training.

Contribution

It introduces an expression-aware conditioning module and a facial image-text alignment system to address common issues in portrait animation, advancing the state-of-the-art in visual coherence and diversity.

Findings

Enhanced identity retention and facial motion transfer accuracy.

Improved temporal coherence and visual stability in animations.

Versatile application potential in virtual avatars and digital media.

Abstract

We present SkyReels-A1, a simple yet effective framework built upon video diffusion Transformer to facilitate portrait image animation. Existing methodologies still encounter issues, including identity distortion, background instability, and unrealistic facial dynamics, particularly in head-only animation scenarios. Besides, extending to accommodate diverse body proportions usually leads to visual inconsistencies or unnatural articulations. To address these challenges, SkyReels-A1 capitalizes on the strong generative capabilities of video DiT, enhancing facial motion transfer precision, identity retention, and temporal coherence. The system incorporates an expression-aware conditioning module that enables seamless video synthesis driven by expression-guided landmark inputs. Integrating the facial image-text alignment module strengthens the fusion of facial attributes with motion trajectories, reinforcing identity preservation. Additionally, SkyReels-A1 incorporates a multi-stage training paradigm to incrementally refine the correlation between expressions and motion while ensuring stable identity reproduction. Extensive empirical evaluations highlight the model's ability to produce visually coherent and compositionally diverse results, making it highly applicable to domains such as virtual avatars, remote communication, and digital media generation.