Geometry-Aware Face Completion and Editing

TL;DR

This paper introduces FCENet, a geometry-aware face completion and editing network that leverages facial geometry estimation and low-rank regularization to produce realistic, editable face images with diverse outputs.

Contribution

The paper proposes a novel geometry-aware framework that estimates facial landmarks and parsing maps to improve face completion and enable attribute editing.

Findings

Achieves visually pleasing face completion results.

Enables diverse face editing by modifying facial geometry.

Outperforms existing methods quantitatively and qualitatively.

Abstract

Face completion is a challenging generation task because it requires generating visually pleasing new pixels that are semantically consistent with the unmasked face region. This paper proposes a geometry-aware Face Completion and Editing NETwork (FCENet) by systematically studying facial geometry from the unmasked region. Firstly, a facial geometry estimator is learned to estimate facial landmark heatmaps and parsing maps from the unmasked face image. Then, an encoder-decoder structure generator serves to complete a face image and disentangle its mask areas conditioned on both the masked face image and the estimated facial geometry images. Besides, since low-rank property exists in manually labeled masks, a low-rank regularization term is imposed on the disentangled masks, enforcing our completion network to manage occlusion area with various shape and size. Furthermore, our network can generate diverse results from the same masked input by modifying estimated facial geometry, which provides a flexible mean to edit the completed face appearance. Extensive experimental results qualitatively and quantitatively demonstrate that our network is able to generate visually pleasing face completion results and edit face attributes as well.