High Resolution Face Editing with Masked GAN Latent Code Optimization

TL;DR

MaskFaceGAN is a novel high-resolution face editing method that optimizes the latent space of StyleGAN2 for local attribute editing, achieving high-quality, artifact-free results with fine-grained control and less attribute entanglement.

Contribution

The paper introduces MaskFaceGAN, a new approach for local face attribute editing that directly optimizes GAN latent codes with spatially selective constraints, improving quality and control.

Findings

Effective local attribute editing at 1024x1024 resolution.

Outperforms state-of-the-art methods in image quality and attribute disentanglement.

Enables precise, artifact-free face modifications with fine-grained control.

Abstract

Face editing represents a popular research topic within the computer vision and image processing communities. While significant progress has been made recently in this area, existing solutions: (i) are still largely focused on low-resolution images, (ii) often generate editing results with visual artefacts, or (iii) lack fine-grained control and alter multiple (entangled) attributes at once, when trying to generate the desired facial semantics. In this paper, we aim to address these issues though a novel attribute editing approach called MaskFaceGAN that focuses on local attribute editing. The proposed approach is based on an optimization procedure that directly optimizes the latent code of a pre-trained (state-of-the-art) Generative Adversarial Network (i.e., StyleGAN2) with respect to several constraints that ensure: (i) preservation of relevant image content, (ii) generation of the targeted facial attributes, and (iii) spatially--selective treatment of local image areas. The constraints are enforced with the help of an (differentiable) attribute classifier and face parser that provide the necessary reference information for the optimization procedure. MaskFaceGAN is evaluated in extensive experiments on the CelebA-HQ, Helen and SiblingsDB-HQf datasets and in comparison with several state-of-the-art techniques from the literature, i.e., StarGAN, AttGAN, STGAN, and two versions of InterFaceGAN. Our experimental results show that the proposed approach is able to edit face images with respect to several local facial attributes with unprecedented image quality and at high-resolutions (1024x1024), while exhibiting considerably less problems with attribute entanglement than competing solutions. The source code is made freely available from: https://github.com/MartinPernus/MaskFaceGAN.