A spin-glass model for the loss surfaces of generative adversarial networks

TL;DR

This paper introduces a spin-glass based mathematical model to analyze the complex loss surfaces of large GANs, providing new theoretical insights into their critical points and structure.

Contribution

It presents a novel spin-glass model for GANs and applies Random Matrix Theory to analyze their loss surface complexity, revealing unique structural insights.

Findings

Insights into the critical points of GAN loss surfaces

Revealed new structural features of large GANs

Extended prior theoretical understanding of neural network loss landscapes

Abstract

We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.