Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space

TL;DR

This paper introduces a Gaussianized latent space model for StyleGAN that improves inversion stability, enhances image diversity, and reduces artifacts by regularizing latent projections with a Gaussian prior.

Contribution

It proposes a simple nonlinear transformation to model latent space as Gaussian, leading to more stable image inversion and artifact reduction in StyleGAN.

Findings

Smoother interpolation between real and generated images

Improved stability in latent space projections

Reduced artifacts while maintaining diversity

Abstract

Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. It has been shown that a wide range of images can be projected into this space, including images outside of the domain that the generator was trained on. However, while in this case the generator reproduces the pixels and textures of the images, the reconstructed latent vectors are unstable and small perturbations result in significant image distortions. In this work, we propose to explicitly model the data distribution in latent space. We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics. This yields a simple Gaussian prior, which we use to regularize the projection of images into the latent space. The resulting projections lie in smoother and better behaved regions of the latent space, as shown using interpolation performance for both real and generated images. Furthermore, the Gaussian model of the distribution in latent space allows us to investigate the origins of artifacts in the generator output, and provides a method for reducing these artifacts while maintaining diversity of the generated images.