Transferring GANs: generating images from limited data

TL;DR

This paper explores how transfer learning techniques can improve image generation in GANs, especially with limited data, by leveraging pretrained models to enhance quality and reduce training time.

Contribution

It is the first study to analyze domain adaptation in generative adversarial networks, demonstrating benefits of transfer learning for image synthesis with limited data.

Findings

Transfer learning shortens GAN training convergence time.

Pretrained models improve image quality in limited data scenarios.

Density of training data impacts transfer effectiveness more than diversity.

Abstract

Transferring the knowledge of pretrained networks to new domains by means of finetuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been studied within the context of generative deep networks. Therefore, we study domain adaptation applied to image generation with generative adversarial networks. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Our results show that using knowledge from pretrained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when the target data is limited. We show that these conclusions can also be drawn for conditional GANs even when the pretrained model was trained without conditioning. Our results also suggest that density may be more important than diversity and a dataset with one or few densely sampled classes may be a better source model than more diverse datasets such as ImageNet or Places.