Facial Attribute Transformers for Precise and Robust Makeup Transfer

TL;DR

This paper introduces Facial Attribute Transformers (FAT) and Spatial FAT, novel models that improve makeup transfer by enhancing color fidelity, geometric transformation, and handling facial variations for high-quality, high-resolution results.

Contribution

The paper presents the first Transformer-based framework for makeup transfer, integrating spatial deformation capabilities with FAT to transfer both color and geometric facial attributes.

Findings

High-fidelity color transfer achieved

Effective geometric transformation of facial parts

Robust handling of facial pose and shadow variations

Abstract

In this paper, we address the problem of makeup transfer, which aims at transplanting the makeup from the reference face to the source face while preserving the identity of the source. Existing makeup transfer methods have made notable progress in generating realistic makeup faces, but do not perform well in terms of color fidelity and spatial transformation. To tackle these issues, we propose a novel Facial Attribute Transformer (FAT) and its variant Spatial FAT for high-quality makeup transfer. Drawing inspirations from the Transformer in NLP, FAT is able to model the semantic correspondences and interactions between the source face and reference face, and then precisely estimate and transfer the facial attributes. To further facilitate shape deformation and transformation of facial parts, we also integrate thin plate splines (TPS) into FAT, thus creating Spatial FAT, which is the first method that can transfer geometric attributes in addition to color and texture. Extensive qualitative and quantitative experiments demonstrate the effectiveness and superiority of our proposed FATs in the following aspects: (1) ensuring high-fidelity color transfer; (2) allowing for geometric transformation of facial parts; (3) handling facial variations (such as poses and shadows) and (4) supporting high-resolution face generation.