Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies

TL;DR

This paper reviews recent regularization techniques for GANs, categorizing them by principles, analyzing their differences, and discussing their practical applications and limitations to improve training stability.

Contribution

It provides a comprehensive overview of recent GAN regularization methods, classifies them by operation principles, and offers insights into their practical use and future research directions.

Findings

Most recent methods published in last three years

Classification of regularization methods by operation principles

Identification of limitations and future research directions

Abstract

Despite its short history, Generative Adversarial Network (GAN) has been extensively studied and used for various tasks, including its original purpose, i.e., synthetic sample generation. However, applying GAN to different data types with diverse neural network architectures has been hindered by its limitation in training, where the model easily diverges. Such a notorious training of GANs is well known and has been addressed in numerous studies. Consequently, in order to make the training of GAN stable, numerous regularization methods have been proposed in recent years. This paper reviews the regularization methods that have been recently introduced, most of which have been published in the last three years. Specifically, we focus on general methods that can be commonly used regardless of neural network architectures. To explore the latest research trends in the regularization for GANs, the methods are classified into several groups by their operation principles, and the differences between the methods are analyzed. Furthermore, to provide practical knowledge of using these methods, we investigate popular methods that have been frequently employed in state-of-the-art GANs. In addition, we discuss the limitations in existing methods and propose future research directions.