Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data

TL;DR

This paper introduces Primitives-PS, a synthetic data generator based on natural image properties, which enhances GAN transfer learning, improves performance across datasets, and reduces privacy risks.

Contribution

The authors propose Primitives-PS, a novel synthetic data synthesizer leveraging natural image characteristics to improve GAN pretraining transferability and privacy safety.

Findings

Pretrained GANs with Primitives-PS outperform previous methods in FID scores.

The synthesizer's components are validated through extensive ablation studies.

The approach ensures better generalization across diverse target datasets.

Abstract

Transfer learning for GANs successfully improves generation performance under low-shot regimes. However, existing studies show that the pretrained model using a single benchmark dataset is not generalized to various target datasets. More importantly, the pretrained model can be vulnerable to copyright or privacy risks as membership inference attack advances. To resolve both issues, we propose an effective and unbiased data synthesizer, namely Primitives-PS, inspired by the generic characteristics of natural images. Specifically, we utilize 1) the generic statistics on the frequency magnitude spectrum, 2) the elementary shape (i.e., image composition via elementary shapes) for representing the structure information, and 3) the existence of saliency as prior. Since our synthesizer only considers the generic properties of natural images, the single model pretrained on our dataset can be consistently transferred to various target datasets, and even outperforms the previous methods pretrained with the natural images in terms of Fr'echet inception distance. Extensive analysis, ablation study, and evaluations demonstrate that each component of our data synthesizer is effective, and provide insights on the desirable nature of the pretrained model for the transferability of GANs.