Tensorizing Generative Adversarial Nets

TL;DR

This paper introduces a tensor-based GAN framework that significantly reduces model parameters while maintaining performance, enabling deployment on resource-limited devices like mobile phones.

Contribution

It proposes a novel tensorized architecture for GANs that drastically decreases parameter count without sacrificing generative quality.

Findings

Achieves up to 35x parameter reduction on MNIST

Maintains comparable sample quality to original GANs

Offers an efficient training algorithm for the tensorized model

Abstract

Generative Adversarial Network (GAN) and its variants exhibit state-of-the-art performance in the class of generative models. To capture higher-dimensional distributions, the common learning procedure requires high computational complexity and a large number of parameters. The problem of employing such massive framework arises when deploying it on a platform with limited computational power such as mobile phones. In this paper, we present a new generative adversarial framework by representing each layer as a tensor structure connected by multilinear operations, aiming to reduce the number of model parameters by a large factor while preserving the generative performance and sample quality. To learn the model, we employ an efficient algorithm which alternatively optimizes both discriminator and generator. Experimental outcomes demonstrate that our model can achieve high compression rate for model parameters up to $35$ times when compared to the original GAN for MNIST dataset.