A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Models

TL;DR

This paper introduces the Tensor Ring Induced Prior (TRIP), a new high-dimensional Gaussian mixture prior for generative models, which enhances performance and flexibility in GANs and VAEs, including conditional generation with missing conditions.

Contribution

The paper proposes TRIP, a novel tensor ring-based Gaussian mixture prior that improves generative model performance and can be integrated into various GAN and VAE architectures.

Findings

TRIP improves Fréchet Inception Distance in GANs.

TRIP enhances Evidence Lower Bound in VAEs.

TRIP enables effective conditional generation with missing conditions.

Abstract

Generative models produce realistic objects in many domains, including text, image, video, and audio synthesis. Most popular models---Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs)---usually employ a standard Gaussian distribution as a prior. Previous works show that the richer family of prior distributions may help to avoid the mode collapse problem in GANs and to improve the evidence lower bound in VAEs. We propose a new family of prior distributions---Tensor Ring Induced Prior (TRIP)---that packs an exponential number of Gaussians into a high-dimensional lattice with a relatively small number of parameters. We show that these priors improve Fr\'echet Inception Distance for GANs and Evidence Lower Bound for VAEs. We also study generative models with TRIP in the conditional generation setup with missing conditions. Altogether, we propose a novel plug-and-play framework for generative models that can be utilized in any GAN and VAE-like architectures.