Generative Face Completion

TL;DR

This paper introduces a deep generative model for face completion that effectively generates realistic missing facial regions, handling large missing areas and semantic variations through a combination of multiple loss functions.

Contribution

It presents a novel face completion algorithm that directly generates missing regions using a neural network trained with combined loss functions, improving over patch-based methods.

Findings

Successfully completes large missing facial regions.

Produces realistic and semantically consistent face images.

Outperforms existing methods in qualitative and quantitative evaluations.

Abstract

In this paper, we propose an effective face completion algorithm using a deep generative model. Different from well-studied background completion, the face completion task is more challenging as it often requires to generate semantically new pixels for the missing key components (e.g., eyes and mouths) that contain large appearance variations. Unlike existing nonparametric algorithms that search for patches to synthesize, our algorithm directly generates contents for missing regions based on a neural network. The model is trained with a combination of a reconstruction loss, two adversarial losses and a semantic parsing loss, which ensures pixel faithfulness and local-global contents consistency. With extensive experimental results, we demonstrate qualitatively and quantitatively that our model is able to deal with a large area of missing pixels in arbitrary shapes and generate realistic face completion results.