Foreground-guided Facial Inpainting with Fidelity Preservation

TL;DR

This paper introduces a foreground-guided facial inpainting method that leverages segmentation masks and a novel loss function to preserve facial feature fidelity, achieving superior qualitative results.

Contribution

It proposes a new facial inpainting framework using foreground masks and a semantic-aware loss to enhance fidelity preservation in facial features.

Findings

Achieved high-fidelity preservation of facial components.

Demonstrated qualitative improvements over state-of-the-art methods.

Performed well quantitatively on CelebA-HQ dataset.

Abstract

Facial image inpainting, with high-fidelity preservation for image realism, is a very challenging task. This is due to the subtle texture in key facial features (component) that are not easily transferable. Many image inpainting techniques have been proposed with outstanding capabilities and high quantitative performances recorded. However, with facial inpainting, the features are more conspicuous and the visual quality of the blended inpainted regions are more important qualitatively. Based on these facts, we design a foreground-guided facial inpainting framework that can extract and generate facial features using convolutional neural network layers. It introduces the use of foreground segmentation masks to preserve the fidelity. Specifically, we propose a new loss function with semantic capability reasoning of facial expressions, natural and unnatural features (make-up). We conduct our experiments using the CelebA-HQ dataset, segmentation masks from CelebAMask-HQ (for foreground guidance) and Quick Draw Mask (for missing regions). Our proposed method achieved comparable quantitative results when compare to the state of the art but qualitatively, it demonstrated high-fidelity preservation of facial components.