FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

TL;DR

FaceDancer introduces a novel face swapping method that effectively preserves pose, expression, and occlusion while transferring identity, using adaptive feature fusion and regularization techniques for high fidelity results.

Contribution

The paper proposes two new modules, AFFA and IFSR, for improved identity transfer and pose preservation in face swapping without segmentation.

Findings

Outperforms state-of-the-art in identity transfer accuracy.

Achieves superior pose and attribute preservation.

Demonstrates robustness across diverse datasets.

Abstract

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identityn transfer, while having significantly better pose preservation than most of the previous methods.