ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

TL;DR

ReliableSwap introduces a novel training supervision method using cycle triplets and synthetic images to improve identity preservation in face swapping, especially when source and target differ, with minimal additional computational cost.

Contribution

The paper proposes ReliableSwap, a new framework that enhances existing face swapping methods by using reliable supervision through cycle triplets and a FixerNet for better lower-face detail preservation.

Findings

Significantly improves identity preservation in face swapping.

Effective in handling source-target identity differences.

Achieves superior results with negligible overhead.

Abstract

Almost all advanced face swapping approaches use reconstruction as the proxy task, i.e., supervision only exists when the target and source belong to the same person. Otherwise, lacking pixel-level supervision, these methods struggle for source identity preservation. This paper proposes to construct reliable supervision, dubbed cycle triplets, which serves as the image-level guidance when the source identity differs from the target one during training. Specifically, we use face reenactment and blending techniques to synthesize the swapped face from real images in advance, where the synthetic face preserves source identity and target attributes. However, there may be some artifacts in such a synthetic face. To avoid the potential artifacts and drive the distribution of the network output close to the natural one, we reversely take synthetic images as input while the real face as reliable supervision during the training stage of face swapping. Besides, we empirically find that the existing methods tend to lose lower-face details like face shape and mouth from the source. This paper additionally designs a FixerNet, providing discriminative embeddings of lower faces as an enhancement. Our face swapping framework, named ReliableSwap, can boost the performance of any existing face swapping network with negligible overhead. Extensive experiments demonstrate the efficacy of our ReliableSwap, especially in identity preservation. The project page is https://reliable-swap.github.io/.