Scalable and Realistic Virtual Try-on Application for Foundation Makeup with Kubelka-Munk Theory

TL;DR

This paper introduces a scalable virtual try-on system for foundation makeup that uses an approximation of Kubelka-Munk theory to achieve realistic color blending and fast image synthesis, validated on real-world images.

Contribution

It presents a novel approximation of Kubelka-Munk theory for faster, realistic foundation makeup synthesis and a scalable framework relying solely on e-commerce product data.

Findings

Outperforms existing techniques in realism and speed

Effective on diverse real-world makeup images

Maintains color blending accuracy across products

Abstract

Augmented reality is revolutionizing beauty industry with virtual try-on (VTO) applications, which empowers users to try a wide variety of products using their phones without the hassle of physically putting on real products. A critical technical challenge in foundation VTO applications is the accurate synthesis of foundation-skin tone color blending while maintaining the scalability of the method across diverse product ranges. In this work, we propose a novel method to approximate well-established Kubelka-Munk (KM) theory for faster image synthesis while preserving foundation-skin tone color blending realism. Additionally, we build a scalable end-to-end framework for realistic foundation makeup VTO solely depending on the product information available on e-commerce sites. We validate our method using real-world makeup images, demonstrating that our framework outperforms other techniques.