Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN

TL;DR

Mask-FPAN is a semi-supervised face parsing framework that effectively handles occlusions and pose variations using de-occlusion modules, 3D modeling, and GANs, achieving state-of-the-art results.

Contribution

It introduces a novel semi-supervised face parsing framework with de-occlusion, 3D modeling, and new datasets, advancing face parsing in challenging real-world scenarios.

Findings

Achieves MIOU improvement from 0.7353 to 0.9013

Addresses occlusion and pose variation challenges effectively

Provides new datasets for face parsing research

Abstract

Fine-grained semantic segmentation of a person's face and head, including facial parts and head components, has progressed a great deal in recent years. However, it remains a challenging task, whereby considering ambiguous occlusions and large pose variations are particularly difficult. To overcome these difficulties, we propose a novel framework termed Mask-FPAN. It uses a de-occlusion module that learns to parse occluded faces in a semi-supervised way. In particular, face landmark localization, face occlusionstimations, and detected head poses are taken into account. A 3D morphable face model combined with the UV GAN improves the robustness of 2D face parsing. In addition, we introduce two new datasets named FaceOccMask-HQ and CelebAMaskOcc-HQ for face paring work. The proposed Mask-FPAN framework addresses the face parsing problem in the wild and shows significant performance improvements with MIOU from 0.7353 to 0.9013 compared to the state-of-the-art on challenging face datasets.