TL;DR
This paper presents a real-time hair segmentation method for mobile devices using a modified MobileNet CNN trained on crowdsourced data, enabling accurate virtual hair color augmentation in augmented reality applications.
Contribution
The paper introduces a lightweight CNN architecture optimized for mobile devices that achieves real-time hair segmentation using noisy crowdsourced data.
Findings
Achieves over 30 fps on an iPad Pro
Produces accurate, fine-detailed hair mattes
Uses noisy crowdsourced data effectively
Abstract
Augmented reality is an emerging technology in many application domains. Among them is the beauty industry, where live virtual try-on of beauty products is of great importance. In this paper, we address the problem of live hair color augmentation. To achieve this goal, hair needs to be segmented quickly and accurately. We show how a modified MobileNet CNN architecture can be used to segment the hair in real-time. Instead of training this network using large amounts of accurate segmentation data, which is difficult to obtain, we use crowd sourced hair segmentation data. While such data is much simpler to obtain, the segmentations there are noisy and coarse. Despite this, we show how our system can produce accurate and fine-detailed hair mattes, while running at over 30 fps on an iPad Pro tablet.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
