Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

TL;DR

This paper introduces a three-stage framework for face synthesis from visual attributes, involving sketch generation and face reconstruction using a combination of CVAE and GANs, improving face synthesis quality.

Contribution

It proposes a novel stage-wise approach combining CVAE and GANs for attribute-based face synthesis, which is different from direct image generation methods.

Findings

Effective face synthesis from attributes demonstrated

Enhanced sketch quality using GAN-based refinement

Superior results compared to recent methods

Abstract

Automatic synthesis of faces from visual attributes is an important problem in computer vision and has wide applications in law enforcement and entertainment. With the advent of deep generative convolutional neural networks (CNNs), attempts have been made to synthesize face images from attributes and text descriptions. In this paper, we take a different approach, where we formulate the original problem as a stage-wise learning problem. We first synthesize the facial sketch corresponding to the visual attributes and then we reconstruct the face image based on the synthesized sketch. The proposed Attribute2Sketch2Face framework, which is based on a combination of deep Conditional Variational Autoencoder (CVAE) and Generative Adversarial Networks (GANs), consists of three stages: (1) Synthesis of facial sketch from attributes using a CVAE architecture, (2) Enhancement of coarse sketches to produce sharper sketches using a GAN-based framework, and (3) Synthesis of face from sketch using another GAN-based network. Extensive experiments and comparison with recent methods are performed to verify the effectiveness of the proposed attribute-based three stage face synthesis method.