FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen

TL;DR
FaceShifter introduces a two-stage face swapping framework that achieves high fidelity and occlusion awareness by integrating target attributes adaptively and refining results with a self-supervised error correction network.
Contribution
The paper presents a novel two-stage face swapping method with an attribute encoder, adaptive denormalization, and a self-supervised refinement network for occlusion handling.
Findings
Outperforms state-of-the-art methods in perceptual quality.
Better identity preservation in face swapping.
Effective occlusion handling with HEAR-Net.
Abstract
In this work, we propose a novel two-stage framework, called FaceShifter, for high fidelity and occlusion aware face swapping. Unlike many existing face swapping works that leverage only limited information from the target image when synthesizing the swapped face, our framework, in its first stage, generates the swapped face in high-fidelity by exploiting and integrating the target attributes thoroughly and adaptively. We propose a novel attributes encoder for extracting multi-level target face attributes, and a new generator with carefully designed Adaptive Attentional Denormalization (AAD) layers to adaptively integrate the identity and the attributes for face synthesis. To address the challenging facial occlusions, we append a second stage consisting of a novel Heuristic Error Acknowledging Refinement Network (HEAR-Net). It is trained to recover anomaly regions in a self-supervised…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Generative Adversarial Networks and Image Synthesis · Video Surveillance and Tracking Methods
