A New Distributed Method for Training Generative Adversarial Networks

TL;DR

This paper introduces a novel distributed training framework for GANs that enhances convergence speed by enabling multiple devices to collaboratively train a global model without centralized data aggregation.

Contribution

It proposes a new distributed GAN training method with two update schedules, improving convergence speed over existing frameworks.

Findings

Outperforms state-of-the-art in convergence speed

Effective in distributed environments with privacy constraints

Demonstrated on three popular datasets

Abstract

Generative adversarial networks (GANs) are emerging machine learning models for generating synthesized data similar to real data by jointly training a generator and a discriminator. In many applications, data and computational resources are distributed over many devices, so centralized computation with all data in one location is infeasible due to privacy and/or communication constraints. This paper proposes a new framework for training GANs in a distributed fashion: Each device computes a local discriminator using local data; a single server aggregates their results and computes a global GAN. Specifically, in each iteration, the server sends the global GAN to the devices, which then update their local discriminators; the devices send their results to the server, which then computes their average as the global discriminator and updates the global generator accordingly. Two different update schedules are designed with different levels of parallelism between the devices and the server. Numerical results obtained using three popular datasets demonstrate that the proposed framework can outperform a state-of-the-art framework in terms of convergence speed.