Face Mask Removal with Region-attentive Face Inpainting

TL;DR

This paper introduces a novel face inpainting method that effectively removes masks from face images, preserving identity and high fidelity, using a multi-scale attention module and supervised focus on masked regions.

Contribution

The paper proposes a region-attentive face inpainting approach with a new multi-scale attention module and a supervised mechanism to focus on masked areas, along with a new Masked-Faces dataset.

Findings

Outperforms baselines in SSIM, PSNR, and L1 loss.

Generates high-quality, identity-preserving face reconstructions.

Provides publicly available code and dataset.

Abstract

During the COVID-19 pandemic, face masks have become ubiquitous in our lives. Face masks can cause some face recognition models to fail since they cover significant portion of a face. In addition, removing face masks from captured images or videos can be desirable, e.g., for better social interaction and for image/video editing and enhancement purposes. Hence, we propose a generative face inpainting method to effectively recover/reconstruct the masked part of a face. Face inpainting is more challenging compared to traditional inpainting, since it requires high fidelity while maintaining the identity at the same time. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss and learn the inter- and intra-channel correlation. In addition, we introduce an approach enforcing the supervised signal to focus on masked regions instead of the whole image. We also synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks, including surgical mask, regular mask and scarves, which also cover the neck area. The experimental results show that our proposed method outperforms different baselines in terms of structural similarity index measure, peak signal-to-noise ratio and l1 loss, while also providing better outputs qualitatively. The code will be made publicly available. Code is available at GitHub.