Introspective Generative Modeling: Decide Discriminatively

TL;DR

This paper introduces introspective generative modeling (IGM), a novel approach where a generator also functions as a discriminator, enabling self-evaluation and improved learning for tasks like texture synthesis, style transfer, and face modeling.

Contribution

The paper presents a new unsupervised learning framework where the generator is a discriminator that self-evaluates, inheriting properties of discriminative classifiers through a cascade of CNNs.

Findings

Encouraging results in texture modeling

Effective artistic style transfer

Improved face modeling

Abstract

We study unsupervised learning by developing introspective generative modeling (IGM) that attains a generator using progressively learned deep convolutional neural networks. The generator is itself a discriminator, capable of introspection: being able to self-evaluate the difference between its generated samples and the given training data. When followed by repeated discriminative learning, desirable properties of modern discriminative classifiers are directly inherited by the generator. IGM learns a cascade of CNN classifiers using a synthesis-by-classification algorithm. In the experiments, we observe encouraging results on a number of applications including texture modeling, artistic style transferring, face modeling, and semi-supervised learning.