Discovering Class-Specific GAN Controls for Semantic Image Synthesis

TL;DR

This paper introduces a new optimization method to discover class-specific latent directions in pretrained semantic image synthesis GANs, enabling precise local control over semantic classes for improved image editing.

Contribution

It presents a novel approach for finding spatially disentangled, class-specific latent directions in pretrained SIS models, facilitating targeted semantic edits.

Findings

Discovered diverse, semantically meaningful latent directions for class-specific control.

Effective local control over appearance, texture, and color of semantic classes.

Quantitative and visual evaluations confirm the method's ability to produce meaningful edits.

Abstract

Prior work has extensively studied the latent space structure of GANs for unconditional image synthesis, enabling global editing of generated images by the unsupervised discovery of interpretable latent directions. However, the discovery of latent directions for conditional GANs for semantic image synthesis (SIS) has remained unexplored. In this work, we specifically focus on addressing this gap. We propose a novel optimization method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes, e.g., changing their internal structure, texture or color independently from each other. Visual inspection and quantitative evaluation of the discovered GAN controls on various datasets demonstrate that our method discovers a diverse set of unique and semantically meaningful latent directions for class-specific edits.