Multimodal Face Synthesis from Visual Attributes

TL;DR

This paper introduces a novel GAN framework that synthesizes multimodal face images from visual attributes, preserving identity across modalities without needing paired training data, advancing face synthesis technology.

Contribution

A new GAN architecture with multimodal stretch-out and stretch-in modules enables identity-preserving multimodal face synthesis from attributes without paired data.

Findings

Effective multimodal face synthesis demonstrated

Outperforms state-of-the-art methods

Preserves identity across modalities

Abstract

Synthesis of face images from visual attributes is an important problem in computer vision and biometrics due to its applications in law enforcement and entertainment. Recent advances in deep generative networks have made it possible to synthesize high-quality face images from visual attributes. However, existing methods are specifically designed for generating unimodal images (i.e visible faces) from attributes. In this paper, we propose a novel generative adversarial network that simultaneously synthesizes identity preserving multimodal face images (i.e. visible, sketch, thermal, etc.) from visual attributes without requiring paired data in different domains for training the network. We introduce a novel generator with multimodal stretch-out modules to simultaneously synthesize multimodal face images. Additionally, multimodal stretch-in modules are introduced in the discriminator which discriminates between real and fake images. Extensive experiments and comparisons with several state-of-the-art methods are performed to verify the effectiveness of the proposed attribute-based multimodal synthesis method.