FlowFace: Semantic Flow-guided Shape-aware Face Swapping

TL;DR

FlowFace introduces a two-stage face swapping framework that effectively transfers both facial shape and inner features, resulting in more realistic and identity-preserving face swaps by leveraging semantic flow and a pre-trained autoencoder.

Contribution

The paper presents a novel semantic flow-guided approach for shape-aware face swapping, explicitly transferring facial contours and inner features with a new reshaping and fusion method.

Findings

Outperforms state-of-the-art face swapping methods

Effectively transfers facial shape and identity

Produces more realistic face swaps

Abstract

In this work, we propose a semantic flow-guided two-stage framework for shape-aware face swapping, namely FlowFace. Unlike most previous methods that focus on transferring the source inner facial features but neglect facial contours, our FlowFace can transfer both of them to a target face, thus leading to more realistic face swapping. Concretely, our FlowFace consists of a face reshaping network and a face swapping network. The face reshaping network addresses the shape outline differences between the source and target faces. It first estimates a semantic flow (i.e., face shape differences) between the source and the target face, and then explicitly warps the target face shape with the estimated semantic flow. After reshaping, the face swapping network generates inner facial features that exhibit the identity of the source face. We employ a pre-trained face masked autoencoder (MAE) to extract facial features from both the source face and the target face. In contrast to previous methods that use identity embedding to preserve identity information, the features extracted by our encoder can better capture facial appearances and identity information. Then, we develop a cross-attention fusion module to adaptively fuse inner facial features from the source face with the target facial attributes, thus leading to better identity preservation. Extensive quantitative and qualitative experiments on in-the-wild faces demonstrate that our FlowFace outperforms the state-of-the-art significantly.