Disentangled Makeup Transfer with Generative Adversarial Network

TL;DR

This paper introduces DMT, a GAN-based model that disentangles identity and makeup style to enable flexible, controllable, and high-quality makeup transfer across diverse face images.

Contribution

The paper proposes a novel disentangled representation framework for makeup transfer, allowing for multiple transfer scenarios and style sampling, improving flexibility and quality.

Findings

Achieves high-quality makeup transfer with controllable strength.

Supports multiple scenarios including style transfer and sampling.

Outperforms existing methods in visual quality and versatility.

Abstract

Facial makeup transfer is a widely-used technology that aims to transfer the makeup style from a reference face image to a non-makeup face. Existing literature leverage the adversarial loss so that the generated faces are of high quality and realistic as real ones, but are only able to produce fixed outputs. Inspired by recent advances in disentangled representation, in this paper we propose DMT (Disentangled Makeup Transfer), a unified generative adversarial network to achieve different scenarios of makeup transfer. Our model contains an identity encoder as well as a makeup encoder to disentangle the personal identity and the makeup style for arbitrary face images. Based on the outputs of the two encoders, a decoder is employed to reconstruct the original faces. We also apply a discriminator to distinguish real faces from fake ones. As a result, our model can not only transfer the makeup styles from one or more reference face images to a non-makeup face with controllable strength, but also produce various outputs with styles sampled from a prior distribution. Extensive experiments demonstrate that our model is superior to existing literature by generating high-quality results for different scenarios of makeup transfer.