Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures

TL;DR

This paper demonstrates using random matrix theory that deep learning representations of GAN-generated data behave like Gaussian mixtures, enabling simplified statistical descriptions for classification tasks.

Contribution

It introduces a theoretical framework showing GAN data representations are concentrated vectors that asymptotically resemble Gaussian mixtures, validated through experiments with BigGAN and deep networks.

Findings

GAN representations are concentrated random vectors.

Gram matrices behave as Gaussian mixtures asymptotically.

Deep representations can be characterized by their first two moments.

Abstract

This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called \textit{concentrated} random vectors. Further exploiting the fact that Gram matrices, of the type $G = X^T X$ with $X=[x_1,\ldots,x_n]\in \mathbb{R}^{p\times n}$ and $x_i$ independent concentrated random vectors from a mixture model, behave asymptotically (as $n,p\to \infty$) as if the $x_i$ were drawn from a Gaussian mixture, suggests that DL representations of GAN-data can be fully described by their first two statistical moments for a wide range of standard classifiers. Our theoretical findings are validated by generating images with the BigGAN model and across different popular deep representation networks.