Real-time Segmentation and Facial Skin Tones Grading

TL;DR

This paper introduces an efficient deep learning-based method for real-time facial skin and hair segmentation, achieving high accuracy and speed, and utilizing color features for skin tone grading, suitable for CPU environments.

Contribution

The authors propose a lightweight segmentation network (HLNet) that balances speed and accuracy and incorporate color moment features for skin tone classification, demonstrating effectiveness on multiple datasets.

Findings

Achieves 90.73% pixel accuracy on Figaro1k dataset at 16 FPS on CPU.

Effective skin tone grading with approximately 80% accuracy.

Model generalizes well across different datasets.

Abstract

Modern approaches for semantic segmention usually pay too much attention to the accuracy of the model, and therefore it is strongly recommended to introduce cumbersome backbones, which brings heavy computation burden and memory footprint. To alleviate this problem, we propose an efficient segmentation method based on deep convolutional neural networks (DCNNs) for the task of hair and facial skin segmentation, which achieving remarkable trade-off between speed and performance on three benchmark datasets. As far as we know, the accuracy of skin tones classification is usually unsatisfactory due to the influence of external environmental factors such as illumination and background noise. Therefore, we use the segmentated face to obtain a specific face area, and further exploit the color moment algorithm to extract its color features. Specifically, for a 224 x 224 standard input, using our high-resolution spatial detail information and low-resolution contextual information fusion network (HLNet), we achieve 90.73% Pixel Accuracy on Figaro1k dataset at over 16 FPS in the case of CPU environment. Additional experiments on CamVid dataset further confirm the universality of the proposed model. We further use masked color moment for skin tones grade evaluation and approximate 80% classification accuracy demonstrate the feasibility of the proposed scheme.Code is available at https://github.com/JACKYLUO1991/Face-skin-hair-segmentaiton-and-skin-color-evaluation.