From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data

TL;DR

This paper introduces a new pipeline and framework for makeup transfer that enhances identity preservation and domain adaptation from synthetic to real data, supported by a diverse new benchmark.

Contribution

It proposes ConsistentBeauty for high-fidelity synthetic data curation and RealBeauty for synthetic-to-real model adaptation, improving makeup transfer performance.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Demonstrates improved identity preservation in complex real-world scenarios.

Establishes a diverse new benchmark for makeup transfer evaluation.

Abstract

Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background. Early methods formulate this task as unsupervised image-to-image translation, relying on surrogate objectives and often yielding limited performance. Recent diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements. However, these methods still face two critical challenges: synthetic supervision frequently fails to faithfully preserve identity, and the domain gap between synthetic and real data limits generalization, resulting in degraded performance in complex real-world scenarios. To address these issues, this paper first proposes ConsistentBeauty, a novel data curation pipeline that ensures makeup fidelity and strict identity consistency within the synthesized data. Second, we propose RealBeauty, a synthetic-to-real post-training framework. Beyond supervised learning on curated synthetic data, we further adapt the model to real-world scenarios through reinforcement learning and design novel verifiable rewards tailored to the makeup transfer task. It allows the model to further benefit from real makeup patterns beyond synthetic supervision. In addition, we establish a new diverse benchmark for makeup transfer, covering a wide range of skin tones, ages, genders, poses, and makeup styles, thereby enabling a more comprehensive evaluation of model performance under diverse real-world conditions. Extensive experiments show that our method achieves state-of-the-art performance on multiple benchmarks and demonstrates clear advantages in identity preservation and performance on complex real-world cases.