GAT-GMM: Generative Adversarial Training for Gaussian Mixture Models

TL;DR

This paper introduces GAT-GMM, a novel GAN framework designed to effectively learn Gaussian mixture models by leveraging optimal transport theory and a specialized minimax optimization approach, achieving results comparable to EM algorithms.

Contribution

GAT-GMM is the first GAN-based method tailored for GMMs, utilizing a new zero-sum game with a linear generator and quadratic discriminator, and proven to converge to true parameters in certain cases.

Findings

GAT-GMM converges to an approximate stationary point.

In symmetric two-Gaussian cases, it recovers true parameters.

Performs comparably to EM in experiments.

Abstract

Generative adversarial networks (GANs) learn the distribution of observed samples through a zero-sum game between two machine players, a generator and a discriminator. While GANs achieve great success in learning the complex distribution of image, sound, and text data, they perform suboptimally in learning multi-modal distribution-learning benchmarks including Gaussian mixture models (GMMs). In this paper, we propose Generative Adversarial Training for Gaussian Mixture Models (GAT-GMM), a minimax GAN framework for learning GMMs. Motivated by optimal transport theory, we design the zero-sum game in GAT-GMM using a random linear generator and a softmax-based quadratic discriminator architecture, which leads to a non-convex concave minimax optimization problem. We show that a Gradient Descent Ascent (GDA) method converges to an approximate stationary minimax point of the GAT-GMM optimization problem. In the benchmark case of a mixture of two symmetric, well-separated Gaussians, we further show this stationary point recovers the true parameters of the underlying GMM. We numerically support our theoretical findings by performing several experiments, which demonstrate that GAT-GMM can perform as well as the expectation-maximization algorithm in learning mixtures of two Gaussians.