Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfer

TL;DR

This paper introduces a comprehensive makeup transfer framework capable of handling diverse makeup styles, including colors, patterns, and textures, surpassing previous color-only methods, and demonstrates state-of-the-art results on new datasets.

Contribution

A holistic makeup transfer system with improved color and novel pattern transfer branches, enabling realistic application of diverse makeup styles.

Findings

Achieves state-of-the-art performance on both light and extreme makeup styles.

Introduces new datasets for real and synthetic extreme makeup.

Effectively transfers colors, patterns, and textures in makeup images.

Abstract

Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but also patterns, such as stickers, blushes, and jewelries. However, existing works overlooked the latter components and confined makeup transfer to color manipulation, focusing only on light makeup styles. In this work, we propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeup properties, including color, shape, texture, and location. To train and evaluate such a system, we also introduce new makeup datasets for real and synthetic extreme makeup. Experimental results show that our framework achieves the state of the art performance on both light and extreme makeup styles. Code is available at https://github.com/VinAIResearch/CPM.