PortraitGAN for Flexible Portrait Manipulation

TL;DR

PortraitGAN introduces a flexible, multi-modality portrait manipulation framework that enables continuous and realistic facial edits by leveraging adversarial learning, cycle-consistency, and facial landmarks.

Contribution

It is the first to combine cycle-consistency with multi-modality and continuous facial attribute editing in a unified adversarial framework.

Findings

Effective in continuous facial attribute editing

Supports multi-modality portrait manipulation

Produces photo-realistic results

Abstract

Previous methods have dealt with discrete manipulation of facial attributes such as smile, sad, angry, surprise etc, out of canonical expressions and they are not scalable, operating in single modality. In this paper, we propose a novel framework that supports continuous edits and multi-modality portrait manipulation using adversarial learning. Specifically, we adapt cycle-consistency into the conditional setting by leveraging additional facial landmarks information. This has two effects: first cycle mapping induces bidirectional manipulation and identity preserving; second pairing samples from different modalities can thus be utilized. To ensure high-quality synthesis, we adopt texture-loss that enforces texture consistency and multi-level adversarial supervision that facilitates gradient flow. Quantitative and qualitative experiments show the effectiveness of our framework in performing flexible and multi-modality portrait manipulation with photo-realistic effects.