BlendFace: Re-designing Identity Encoders for Face-Swapping

TL;DR

BlendFace is a new identity encoder designed for face-swapping that reduces attribute entanglement by training on blended images, leading to improved identity-attribute disentanglement without sacrificing performance.

Contribution

It introduces BlendFace, a face recognition model trained on blended images to mitigate attribute biases in face-swapping tasks.

Findings

Improves identity-attribute disentanglement in face-swapping.

Maintains comparable quantitative performance to existing methods.

Reduces attribute biases caused by pretraining on face recognition tasks.

Abstract

The great advancements of generative adversarial networks and face recognition models in computer vision have made it possible to swap identities on images from single sources. Although a lot of studies seems to have proposed almost satisfactory solutions, we notice previous methods still suffer from an identity-attribute entanglement that causes undesired attributes swapping because widely used identity encoders, eg, ArcFace, have some crucial attribute biases owing to their pretraining on face recognition tasks. To address this issue, we design BlendFace, a novel identity encoder for face-swapping. The key idea behind BlendFace is training face recognition models on blended images whose attributes are replaced with those of another mitigates inter-personal biases such as hairsyles. BlendFace feeds disentangled identity features into generators and guides generators properly as an identity loss function. Extensive experiments demonstrate that BlendFace improves the identity-attribute disentanglement in face-swapping models, maintaining a comparable quantitative performance to previous methods.