Guided Image Generation with Conditional Invertible Neural Networks

TL;DR

This paper introduces a conditional invertible neural network (cINN) architecture for guided natural image generation that produces diverse, sharp images without mode collapse, and allows for latent space manipulation.

Contribution

The paper presents a novel cINN architecture combining generative and feed-forward networks, trained with maximum likelihood, enabling diverse, sharp image generation and latent space exploration.

Findings

cINN generates diverse, sharp images without mode collapse.

The model effectively performs image colorization and digit generation.

Latent space manipulation allows intuitive style changes.

Abstract

In this work, we address the task of natural image generation guided by a conditioning input. We introduce a new architecture called conditional invertible neural network (cINN). The cINN combines the purely generative INN model with an unconstrained feed-forward network, which efficiently preprocesses the conditioning input into useful features. All parameters of the cINN are jointly optimized with a stable, maximum likelihood-based training procedure. By construction, the cINN does not experience mode collapse and generates diverse samples, in contrast to e.g. cGANs. At the same time our model produces sharp images since no reconstruction loss is required, in contrast to e.g. VAEs. We demonstrate these properties for the tasks of MNIST digit generation and image colorization. Furthermore, we take advantage of our bi-directional cINN architecture to explore and manipulate emergent properties of the latent space, such as changing the image style in an intuitive way.