Multi-level Latent Space Structuring for Generative Control

TL;DR

This paper introduces a multi-level truncation method for StyleGAN that improves the balance between image quality and diversity by decomposing the latent space into clusters and customizing truncation at different semantic levels.

Contribution

It presents a novel truncation technique using a learnable mixture of Gaussians to decompose the latent space and enable multi-level control over generated samples.

Findings

Enhanced quality-diversity trade-off compared to existing methods

More faithful reproduction of original untruncated samples

Quantitative and qualitative improvements demonstrated

Abstract

Truncation is widely used in generative models for improving the quality of the generated samples, at the expense of reducing their diversity. We propose to leverage the StyleGAN generative architecture to devise a new truncation technique, based on a decomposition of the latent space into clusters, enabling customized truncation to be performed at multiple semantic levels. We do so by learning to re-generate W-space, the extended intermediate latent space of StyleGAN, using a learnable mixture of Gaussians, while simultaneously training a classifier to identify, for each latent vector, the cluster that it belongs to. The resulting truncation scheme is more faithful to the original untruncated samples and allows a better trade-off between quality and diversity. We compare our method to other truncation approaches for StyleGAN, both qualitatively and quantitatively.