Lyapunov Exponents for Diversity in Differentiable Games

TL;DR

This paper introduces Generalized Ridge Rider (GRR), a novel algorithm inspired by dynamical systems theory, for finding diverse solutions in multi-agent and high-dimensional optimization problems, including GANs.

Contribution

It extends Ridge Rider to non-conservative systems, enabling the discovery of diverse solutions at bifurcation points in complex multi-agent settings.

Findings

Successfully visualized bifurcation phenomena in toy problems.

Demonstrated diversity of solutions in iterated prisoners' dilemma.

Achieved diverse solutions in GAN training scenarios.

Abstract

Ridge Rider (RR) is an algorithm for finding diverse solutions to optimization problems by following eigenvectors of the Hessian ("ridges"). RR is designed for conservative gradient systems (i.e., settings involving a single loss function), where it branches at saddles - easy-to-find bifurcation points. We generalize this idea to non-conservative, multi-agent gradient systems by proposing a method - denoted Generalized Ridge Rider (GRR) - for finding arbitrary bifurcation points. We give theoretical motivation for our method by leveraging machinery from the field of dynamical systems. We construct novel toy problems where we can visualize new phenomena while giving insight into high-dimensional problems of interest. Finally, we empirically evaluate our method by finding diverse solutions in the iterated prisoners' dilemma and relevant machine learning problems including generative adversarial networks.