Mask-aware Photorealistic Face Attribute Manipulation

TL;DR

This paper introduces Mask-Adversarial AutoEncoder (M-AAE), a novel method combining VAE and GAN for photorealistic face attribute manipulation that preserves identity and details while allowing continuous attribute editing.

Contribution

It proposes a new approach that modifies feature maps with minimal pixel changes, reinforced by face recognition and cycle consistency losses, and uses facial masks for background consistency.

Findings

Outperforms prior methods in detail preservation

Generates high-quality, attribute-modified face images

Effectively maintains face identity during manipulation

Abstract

The task of face attribute manipulation has found increasing applications, but still remains challenging with the requirement of editing the attributes of a face image while preserving its unique details. In this paper, we choose to combine the Variational AutoEncoder (VAE) and Generative Adversarial Network (GAN) for photorealistic image generation. We propose an effective method to modify a modest amount of pixels in the feature maps of an encoder, changing the attribute strength continuously without hindering global information. Our training objectives of VAE and GAN are reinforced by the supervision of face recognition loss and cycle consistency loss for faithful preservation of face details. Moreover, we generate facial masks to enforce background consistency, which allows our training to focus on manipulating the foreground face rather than background. Experimental results demonstrate our method, called Mask-Adversarial AutoEncoder (M-AAE), can generate high-quality images with changing attributes and outperforms prior methods in detail preservation.