Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation

TL;DR

This paper introduces a diffusion model-based makeup transfer method that uses a curated dataset and text-guided control to improve diversity, fidelity, and user controllability in facial makeup transformation.

Contribution

It presents a novel curated dataset, a diffusion-based disentanglement framework, and a text-guided control mechanism for enhanced makeup transfer.

Findings

Improved makeup transfer fidelity and diversity

Enhanced identity preservation during transformation

Effective region-specific makeup editing with natural language

Abstract

Diffusion models have recently shown strong progress in generative tasks, offering a more stable alternative to GAN-based approaches for makeup transfer. Existing methods often suffer from limited datasets, poor disentanglement between identity and makeup features, and weak controllability. To address these issues, we make three contributions. First, we construct a curated high-quality dataset using a train-generate-filter-retrain strategy that combines synthetic, realistic, and filtered samples to improve diversity and fidelity. Second, we design a diffusion-based framework that disentangles identity and makeup features, ensuring facial structure and skin tone are preserved while applying accurate and diverse cosmetic styles. Third, we propose a text-guided mechanism that allows fine-grained and region-specific control, enabling users to modify eyes, lips, or face makeup with natural language prompts. Experiments on benchmarks and real-world scenarios demonstrate improvements in fidelity, identity preservation, and flexibility. Examples of our dataset can be found at: https://makeup-adapter.github.io.