Visualize and Paint GAN Activations

TL;DR

This paper explores how GAN activations relate to generated structures, enabling control over image synthesis by painting structures with unconditionally trained GANs through the concept of tileable features.

Contribution

It introduces the concept of tileable features to identify activations suitable for painting, enhancing control over GAN-generated images without requiring segmentation during training.

Findings

Identified activations that correlate with generated structures.

Enabled painting of structures using unconditionally trained GANs.

Improved control over image generation from semantic maps.

Abstract

We investigate how generated structures of GANs correlate with their activations in hidden layers, with the purpose of better understanding the inner workings of those models and being able to paint structures with unconditionally trained GANs. This gives us more control over the generated images, allowing to generate them from a semantic segmentation map while not requiring such a segmentation in the training data. To this end we introduce the concept of tileable features, allowing us to identify activations that work well for painting.