Hellinger loss function for Generative Adversarial Networks

TL;DR

This paper introduces Hellinger-based loss functions for GANs, providing theoretical guarantees and empirical evidence of improved robustness and accuracy over traditional methods.

Contribution

It proposes novel Hellinger-type loss functions for GAN training, with theoretical analysis and practical implementations demonstrating enhanced robustness.

Findings

Hellinger loss functions improve robustness against data contamination.

Theoretical properties like consistency and asymptotic normality are established.

Empirical results show better estimation accuracy with Hellinger losses.

Abstract

We propose Hellinger-type loss functions for training Generative Adversarial Networks (GANs), motivated by the boundedness, symmetry, and robustness properties of the Hellinger distance. We define an adversarial objective based on this divergence and study its statistical properties within a general parametric framework. We establish the existence, uniqueness, consistency, and joint asymptotic normality of the estimators obtained from the adversarial training procedure. In particular, we analyze the joint estimation of both generator and discriminator parameters, offering a comprehensive asymptotic characterization of the resulting estimators. We introduce two implementations of the Hellinger-type loss and we evaluate their empirical behavior in comparison with the classic (Maximum Likelihood-type) GAN loss. Through a controlled simulation study, we demonstrate that both proposed losses yield improved estimation accuracy and robustness under increasing levels of data contamination.