A Large-Scale Study on Regularization and Normalization in GANs

TL;DR

This paper provides a comprehensive analysis of regularization and normalization techniques in GANs, highlighting practical challenges, reproducibility issues, and offering open-source tools and models for better understanding and development.

Contribution

It offers an empirical evaluation of various regularization and normalization methods in GANs, addressing reproducibility and practical training issues.

Findings

Identifies common pitfalls in GAN training

Evaluates effectiveness of different regularization schemes

Provides open-source code and pre-trained models

Abstract

Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully applied to many problems, training a GAN is a notoriously challenging task and requires a significant number of hyperparameter tuning, neural architecture engineering, and a non-trivial amount of "tricks". The success in many practical applications coupled with the lack of a measure to quantify the failure modes of GANs resulted in a plethora of proposed losses, regularization and normalization schemes, as well as neural architectures. In this work we take a sober view of the current state of GANs from a practical perspective. We discuss and evaluate common pitfalls and reproducibility issues, open-source our code on Github, and provide pre-trained models on TensorFlow Hub.