MFIM: Megapixel Facial Identity Manipulation

TL;DR

MFIM introduces a high-quality, megapixel face-swapping framework that leverages pretrained StyleGAN and 3DMM for effective identity transformation and offers a novel ID mixing operation for customizable identities.

Contribution

The paper presents a novel face-swapping method that generates megapixel images using StyleGAN and 3DMM supervision, with a new ID mixing operation for identity customization.

Findings

Achieves state-of-the-art face swapping quality.

Effectively transforms facial identity attributes.

Enables customizable identity creation through ID mixing.

Abstract

Face swapping is a task that changes a facial identity of a given image to that of another person. In this work, we propose a novel face-swapping framework called Megapixel Facial Identity Manipulation (MFIM). The face-swapping model should achieve two goals. First, it should be able to generate a high-quality image. We argue that a model which is proficient in generating a megapixel image can achieve this goal. However, generating a megapixel image is generally difficult without careful model design. Therefore, our model exploits pretrained StyleGAN in the manner of GAN-inversion to effectively generate a megapixel image. Second, it should be able to effectively transform the identity of a given image. Specifically, it should be able to actively transform ID attributes (e.g., face shape and eyes) of a given image into those of another person, while preserving ID-irrelevant attributes (e.g., pose and expression). To achieve this goal, we exploit 3DMM that can capture various facial attributes. Specifically, we explicitly supervise our model to generate a face-swapped image with the desirable attributes using 3DMM. We show that our model achieves state-of-the-art performance through extensive experiments. Furthermore, we propose a new operation called ID mixing, which creates a new identity by semantically mixing the identities of several people. It allows the user to customize the new identity.