Spider GAN: Leveraging Friendly Neighbors to Accelerate GAN Training

TL;DR

Spider GAN introduces a novel training method leveraging related datasets as inputs, which accelerates GAN training and improves quality, demonstrated across multiple architectures and datasets with faster convergence and state-of-the-art results.

Contribution

The paper proposes Spider GAN, a new training framework using images as inputs and a measure called SID to identify friendly datasets, leading to faster GAN convergence and improved performance.

Findings

Faster convergence of GAN training with Spider GAN.

Achieved state-of-the-art FID scores with fewer training iterations.

Effective transfer learning via cascaded Spider GAN.

Abstract

Training Generative adversarial networks (GANs) stably is a challenging task. The generator in GANs transform noise vectors, typically Gaussian distributed, into realistic data such as images. In this paper, we propose a novel approach for training GANs with images as inputs, but without enforcing any pairwise constraints. The intuition is that images are more structured than noise, which the generator can leverage to learn a more robust transformation. The process can be made efficient by identifying closely related datasets, or a ``friendly neighborhood'' of the target distribution, inspiring the moniker, Spider GAN. To define friendly neighborhoods leveraging proximity between datasets, we propose a new measure called the signed inception distance (SID), inspired by the polyharmonic kernel. We show that the Spider GAN formulation results in faster convergence, as the generator can discover correspondence even between seemingly unrelated datasets, for instance, between Tiny-ImageNet and CelebA faces. Further, we demonstrate cascading Spider GAN, where the output distribution from a pre-trained GAN generator is used as the input to the subsequent network. Effectively, transporting one distribution to another in a cascaded fashion until the target is learnt -- a new flavor of transfer learning. We demonstrate the efficacy of the Spider approach on DCGAN, conditional GAN, PGGAN, StyleGAN2 and StyleGAN3. The proposed approach achieves state-of-the-art Frechet inception distance (FID) values, with one-fifth of the training iterations, in comparison to their baseline counterparts on high-resolution small datasets such as MetFaces, Ukiyo-E Faces and AFHQ-Cats.