StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation

TL;DR

This paper introduces StyleDomain, a set of efficient, lightweight parameterizations for StyleGAN that improve one-shot and few-shot domain adaptation, enabling better performance with fewer training parameters and revealing new properties of StyleSpace.

Contribution

It proposes novel StyleDomain parameterizations, including Affine+ and AffineLight+, that outperform existing methods in few-shot domain adaptation with fewer parameters.

Findings

StyleDomain directions are sufficient for similar domain adaptation.

Affine+ and AffineLight+ outperform baselines in few-shot scenarios.

Discovered properties of StyleSpace enable domain mixing and image morphing.

Abstract

Domain adaptation of GANs is a problem of fine-tuning GAN models pretrained on a large dataset (e.g. StyleGAN) to a specific domain with few samples (e.g. painting faces, sketches, etc.). While there are many methods that tackle this problem in different ways, there are still many important questions that remain unanswered. In this paper, we provide a systematic and in-depth analysis of the domain adaptation problem of GANs, focusing on the StyleGAN model. We perform a detailed exploration of the most important parts of StyleGAN that are responsible for adapting the generator to a new domain depending on the similarity between the source and target domains. As a result of this study, we propose new efficient and lightweight parameterizations of StyleGAN for domain adaptation. Particularly, we show that there exist directions in StyleSpace (StyleDomain directions) that are sufficient for adapting to similar domains. For dissimilar domains, we propose Affine+ and AffineLight+ parameterizations that allows us to outperform existing baselines in few-shot adaptation while having significantly less training parameters. Finally, we examine StyleDomain directions and discover their many surprising properties that we apply for domain mixing and cross-domain image morphing. Source code can be found at https://github.com/AIRI-Institute/StyleDomain.