DynamicFace: High-Quality and Consistent Face Swapping for Image and Video using Composable 3D Facial Priors

TL;DR

DynamicFace introduces a novel face swapping method using 3D facial priors and diffusion models, achieving high-quality, consistent results in images and videos with improved identity preservation and expression accuracy.

Contribution

It presents a new approach combining 3D priors, diffusion models, and plug-and-play attention layers for precise, disentangled control in face swapping, outperforming previous methods.

Findings

State-of-the-art results on FF++ dataset

Superior image quality and identity preservation

Effective in both image and video face swapping

Abstract

Face swapping transfers the identity of a source face to a target face while retaining the attributes like expression, pose, hair, and background of the target face. Advanced face swapping methods have achieved attractive results. However, these methods often inadvertently transfer identity information from the target face, compromising expression-related details and accurate identity. We propose a novel method DynamicFace that leverages the power of diffusion models and plug-and-play adaptive attention layers for image and video face swapping. First, we introduce four fine-grained facial conditions using 3D facial priors. All conditions are designed to be disentangled from each other for precise and unique control. Then, we adopt Face Former and ReferenceNet for high-level and detailed identity injection. Through experiments on the FF++ dataset, we demonstrate that our method achieves state-of-the-art results in face swapping, showcasing superior image quality, identity preservation, and expression accuracy. Our framework seamlessly adapts to both image and video domains. Our code and results will be available on the project page: https://dynamic-face.github.io/