DynaGAN: Dynamic Few-shot Adaptation of GANs to Multiple Domains

TL;DR

DynaGAN introduces a lightweight hyper-network-based approach for efficient multi-domain few-shot GAN adaptation, leveraging shared knowledge and tensor decomposition to outperform separate models in quality and resource usage.

Contribution

The paper presents DynaGAN, a novel method that enables dynamic, multi-domain GAN adaptation using a hyper-network and tensor decomposition, reducing computational costs and sharing knowledge.

Findings

DynaGAN outperforms separate models in quality and efficiency.

The hyper-network effectively adapts GANs to multiple domains.

Tensor decomposition reduces adaptation complexity.

Abstract

Few-shot domain adaptation to multiple domains aims to learn a complex image distribution across multiple domains from a few training images. A na\"ive solution here is to train a separate model for each domain using few-shot domain adaptation methods. Unfortunately, this approach mandates linearly-scaled computational resources both in memory and computation time and, more importantly, such separate models cannot exploit the shared knowledge between target domains. In this paper, we propose DynaGAN, a novel few-shot domain-adaptation method for multiple target domains. DynaGAN has an adaptation module, which is a hyper-network that dynamically adapts a pretrained GAN model into the multiple target domains. Hence, we can fully exploit the shared knowledge across target domains and avoid the linearly-scaled computational requirements. As it is still computationally challenging to adapt a large-size GAN model, we design our adaptation module light-weight using the rank-1 tensor decomposition. Lastly, we propose a contrastive-adaptation loss suitable for multi-domain few-shot adaptation. We validate the effectiveness of our method through extensive qualitative and quantitative evaluations.