DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations

TL;DR

DeX-Portrait introduces a novel portrait animation method that achieves high-fidelity, disentangled control over head pose and facial expression from a single image and driving video, enabling more precise editing and animation.

Contribution

It presents a new approach combining explicit pose and implicit expression representations with a dual-branch conditioning mechanism for improved disentangled control.

Findings

Outperforms state-of-the-art methods in animation quality

Achieves better disentangled control over pose and expression

Maintains high identity fidelity during animation

Abstract

Portrait animation from a single source image and a driving video is a long-standing problem. Recent approaches tend to adopt diffusion-based image/video generation models for realistic and expressive animation. However, none of these diffusion models realizes high-fidelity disentangled control between the head pose and facial expression, hindering applications like expression-only or pose-only editing and animation. To address this, we propose DeX-Portrait, a novel approach capable of generating expressive portrait animation driven by disentangled pose and expression signals. Specifically, we represent the pose as an explicit global transformation and the expression as an implicit latent code. First, we design a powerful motion trainer to learn both pose and expression encoders for extracting precise and decomposed driving signals. Then we propose to inject the pose transformation into the diffusion model through a dual-branch conditioning mechanism, and the expression latent through cross attention. Finally, we design a progressive hybrid classifier-free guidance for more faithful identity consistency. Experiments show that our method outperforms state-of-the-art baselines on both animation quality and disentangled controllability.