Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing

TL;DR

StyleMapGAN introduces a spatially variant modulation in the latent space of GANs, enabling more accurate and efficient real-time image editing by improving embedding quality and maintaining GAN properties.

Contribution

It proposes a novel spatially variant modulation in the latent space, enhancing real-time image editing accuracy and efficiency compared to existing methods.

Findings

Outperforms state-of-the-art models in local editing and interpolation

Provides more accurate embedding of real images

Maintains compatibility with traditional GAN editing techniques

Abstract

Generative adversarial networks (GANs) synthesize realistic images from random latent vectors. Although manipulating the latent vectors controls the synthesized outputs, editing real images with GANs suffers from i) time-consuming optimization for projecting real images to the latent vectors, ii) or inaccurate embedding through an encoder. We propose StyleMapGAN: the intermediate latent space has spatial dimensions, and a spatially variant modulation replaces AdaIN. It makes the embedding through an encoder more accurate than existing optimization-based methods while maintaining the properties of GANs. Experimental results demonstrate that our method significantly outperforms state-of-the-art models in various image manipulation tasks such as local editing and image interpolation. Last but not least, conventional editing methods on GANs are still valid on our StyleMapGAN. Source code is available at https://github.com/naver-ai/StyleMapGAN.