Learning Representations for Masked Facial Recovery

TL;DR

This paper presents a specialized GAN-based method for unmasking faces in images, improving face recognition performance by recovering masked faces and preserving identity information.

Contribution

It introduces a novel face unmasking approach using GAN inversion tailored for masked face recovery, enhancing recognition accuracy.

Findings

Effective face unmasking demonstrated through extensive experiments.

Improved face verification performance on benchmark datasets.

Preservation of identity information during face recovery.

Abstract

The pandemic of these very recent years has led to a dramatic increase in people wearing protective masks in public venues. This poses obvious challenges to the pervasive use of face recognition technology that now is suffering a decline in performance. One way to address the problem is to revert to face recovery methods as a preprocessing step. Current approaches to face reconstruction and manipulation leverage the ability to model the face manifold, but tend to be generic. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask. We do so by designing a specialized GAN inversion method, based on an appropriate set of losses for learning an unmasking encoder. With extensive experiments, we show that the approach is effective at unmasking face images. In addition, we also show that the identity information is preserved sufficiently well to improve face verification performance based on several face recognition benchmark datasets.