Region-Aware Face Swapping

TL;DR

This paper introduces RAFSwap, a region-aware face swapping network that combines local and global features with a face mask predictor to produce high-resolution, identity-consistent face swaps outperforming existing methods.

Contribution

The novel RAFSwap network integrates local and global feature modeling with an unsupervised face mask predictor for improved face swapping quality.

Findings

Achieves 96.70 ID retrieval accuracy, outperforming SOTA MegaFS by 5.87.

Demonstrates superior qualitative and quantitative results over state-of-the-art methods.

Effectively generates high-resolution, identity-consistent swapped faces.

Abstract

This paper presents a novel Region-Aware Face Swapping (RAFSwap) network to achieve identity-consistent harmonious high-resolution face generation in a local-global manner: \textbf{1)} Local Facial Region-Aware (FRA) branch augments local identity-relevant features by introducing the Transformer to effectively model misaligned cross-scale semantic interaction. \textbf{2)} Global Source Feature-Adaptive (SFA) branch further complements global identity-relevant cues for generating identity-consistent swapped faces. Besides, we propose a \textit{Face Mask Predictor} (FMP) module incorporated with StyleGAN2 to predict identity-relevant soft facial masks in an unsupervised manner that is more practical for generating harmonious high-resolution faces. Abundant experiments qualitatively and quantitatively demonstrate the superiority of our method for generating more identity-consistent high-resolution swapped faces over SOTA methods, \eg, obtaining 96.70 ID retrieval that outperforms SOTA MegaFS by 5.87$\uparrow$.