Scalable Balanced Training of Conditional Generative Adversarial Neural Networks on Image Data

TL;DR

This paper introduces a distributed, scalable training method for conditional GANs that partitions data by labels and trains multiple generators in parallel, significantly improving image quality and scalability on large datasets.

Contribution

The paper presents a novel distributed training approach for DC-CGANs that enhances scalability and reduces generator-discriminator imbalance through label-based data partitioning and parallel training.

Findings

Significant improvement in inception scores and image quality.

Successful weak scaling on large datasets with up to 1,000 processes and 2,000 GPUs.

Effective training on diverse datasets including ImageNet1k.

Abstract

We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1,000 processes and 2,000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.