InvGAN: Invertible GANs

TL;DR

InvGAN introduces a general, architecture-agnostic framework that jointly trains inference and generative models, enabling effective embedding of real images into GAN latent spaces for various editing and augmentation tasks.

Contribution

The paper presents InvGAN, a novel invertible GAN framework that jointly trains inference and generative models, overcoming dataset and architecture limitations of previous inversion methods.

Findings

Successfully embeds real images into GAN latent space

Enables image editing like inpainting and merging

Demonstrates improved inversion quality across datasets

Abstract

Generation of photo-realistic images, semantic editing and representation learning are a few of many potential applications of high resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, image editing or downstream tasks such as classification can not be done on real images using the GAN latent space. Despite numerous efforts to train an inference model or design an iterative method to invert a pre-trained generator, previous methods are dataset (e.g. human face images) and architecture (e.g. StyleGAN) specific. These methods are nontrivial to extend to novel datasets or architectures. We propose a general framework that is agnostic to architecture and datasets. Our key insight is that, by training the inference and the generative model together, we allow them to adapt to each other and to converge to a better quality model. Our \textbf{InvGAN}, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, interpolation and online data augmentation. We demonstrate this with extensive qualitative and quantitative experiments.