Encoder-Powered Generative Adversarial Networks

TL;DR

EncGAN introduces an encoder-based GAN architecture that effectively learns multi-manifold data structures and shared abstract features, enabling improved disentanglement and style transfer capabilities.

Contribution

This work proposes a novel encoder-powered GAN that models data manifolds and shared features using a single latent space, with a simple regularizer and mathematical validation.

Findings

Successfully learns multi-manifold structures in datasets.

Learns disentangled, shared abstract features.

Enables effective style transfer even on out-of-distribution data.

Abstract

We present an encoder-powered generative adversarial network (EncGAN) that is able to learn both the multi-manifold structure and the abstract features of data. Unlike the conventional decoder-based GANs, EncGAN uses an encoder to model the manifold structure and invert the encoder to generate data. This unique scheme enables the proposed model to exclude discrete features from the smooth structure modeling and learn multi-manifold data without being hindered by the disconnections. Also, as EncGAN requires a single latent space to carry the information for all the manifolds, it builds abstract features shared among the manifolds in the latent space. For an efficient computation, we formulate EncGAN using a simple regularizer, and mathematically prove its validity. We also experimentally demonstrate that EncGAN successfully learns the multi-manifold structure and the abstract features of MNIST, 3D-chair and UT-Zap50k datasets. Our analysis shows that the learned abstract features are disentangled and make a good style-transfer even when the source data is off the trained distribution.