IAN: Combining Generative Adversarial Networks for Imaginative Face Generation

TL;DR

This paper introduces IAN, a novel framework combining multiple GANs with a new regularizer to generate imaginative and unseen face images, enabling creative applications like character design.

Contribution

The paper proposes a cascaded GAN framework called IAN and a K-NN based regularizer for GANs, advancing the ability to generate images beyond the training data.

Findings

Effective in generating imaginative face images

Enables manifold traversing and creative character design

Outperforms baseline GAN models in subjective evaluations

Abstract

Generative Adversarial Networks (GANs) have gained momentum for their ability to model image distributions. They learn to emulate the training set and that enables sampling from that domain and using the knowledge learned for useful applications. Several methods proposed enhancing GANs, including regularizing the loss with some feature matching. We seek to push GANs beyond the data in the training and try to explore unseen territory in the image manifold. We first propose a new regularizer for GAN based on K-nearest neighbor (K-NN) selective feature matching to a target set Y in high-level feature space, during the adversarial training of GAN on the base set X, and we call this novel model K-GAN. We show that minimizing the added term follows from cross-entropy minimization between the distributions of GAN and the set Y. Then, We introduce a cascaded framework for GANs that try to address the task of imagining a new distribution that combines the base set X and target set Y by cascading sampling GANs with translation GANs, and we dub the cascade of such GANs as the Imaginative Adversarial Network (IAN). We conduct an objective and subjective evaluation for different IAN setups in the addressed task and show some useful applications for these IANs, like manifold traversing and creative face generation for characters' design in movies or video games.