Signal Propagation in a Gradient-Based and Evolutionary Learning System

TL;DR

This paper compares two distributed coevolutionary algorithms for training GANs, Lipizzaner and Lipi-Ring, finding similar performance but Lipi-Ring offers significant computational efficiency improvements.

Contribution

Introduces Lipi-Ring, a novel ring topology for distributed GAN training, demonstrating comparable quality with reduced computational time.

Findings

Lipi-Ring achieves 14.2% to 41.2% faster training times.

No significant performance difference between Lipi-Ring and Lipizzaner.

Both methods produce comparable quality generative models.

Abstract

Generative adversarial networks (GANs) exhibit training pathologies that can lead to convergence-related degenerative behaviors, whereas spatially-distributed, coevolutionary algorithms (CEAs) for GAN training, e.g. Lipizzaner, are empirically robust to them. The robustness arises from diversity that occurs by training populations of generators and discriminators in each cell of a toroidal grid. Communication, where signals in the form of parameters of the best GAN in a cell propagate in four directions: North, South, West, and East, also plays a role, by communicating adaptations that are both new and fit. We propose Lipi-Ring, a distributed CEA like Lipizzaner, except that it uses a different spatial topology, i.e. a ring. Our central question is whether the different directionality of signal propagation (effectively migration to one or more neighbors on each side of a cell) meets or exceeds the performance quality and training efficiency of Lipizzaner Experimental analysis on different datasets (i.e, MNIST, CelebA, and COVID-19 chest X-ray images) shows that there are no significant differences between the performances of the trained generative models by both methods. However, Lipi-Ring significantly reduces the computational time (14.2%. . . 41.2%). Thus, Lipi-Ring offers an alternative to Lipizzaner when the computational cost of training matters.