Latent reweighting, an almost free improvement for GANs

TL;DR

This paper introduces latent reweighting for GANs, using an additional network to improve sample quality efficiently by reweighting latent space samples, reducing low-quality outputs without modifying the generator architecture.

Contribution

It proposes a novel latent importance weighting method that enhances GAN sampling quality without architectural changes, reducing computation and improving distribution alignment.

Findings

Effective in synthetic and high-dimensional datasets

Reduces low-quality samples in GAN outputs

Improves Wasserstein distance to target distribution

Abstract

Standard formulations of GANs, where a continuous function deforms a connected latent space, have been shown to be misspecified when fitting different classes of images. In particular, the generator will necessarily sample some low-quality images in between the classes. Rather than modifying the architecture, a line of works aims at improving the sampling quality from pre-trained generators at the expense of increased computational cost. Building on this, we introduce an additional network to predict latent importance weights and two associated sampling methods to avoid the poorest samples. This idea has several advantages: 1) it provides a way to inject disconnectedness into any GAN architecture, 2) since the rejection happens in the latent space, it avoids going through both the generator and the discriminator, saving computation time, 3) this importance weights formulation provides a principled way to reduce the Wasserstein's distance to the target distribution. We demonstrate the effectiveness of our method on several datasets, both synthetic and high-dimensional.