On some theoretical limitations of Generative Adversarial Networks

TL;DR

This paper demonstrates that Generative Adversarial Networks cannot generate heavy-tailed distributions, challenging the common assumption of their universal approximation capabilities due to neural network limitations.

Contribution

It provides a new theoretical result using Extreme Value Theory showing the limitations of GANs in generating certain types of distributions.

Findings

GANs cannot generate heavy-tailed distributions

Theoretical proof based on Extreme Value Theory

Challenges the assumption of GANs' universal approximation ability

Abstract

Generative Adversarial Networks have become a core technique in Machine Learning to generate unknown distributions from data samples. They have been used in a wide range of context without paying much attention to the possible theoretical limitations of those models. Indeed, because of the universal approximation properties of Neural Networks, it is a general assumption that GANs can generate any probability distribution. Recently, people began to question this assumption and this article is in line with this thinking. We provide a new result based on Extreme Value Theory showing that GANs can't generate heavy tailed distributions. The full proof of this result is given.