Quality Evaluation of GANs Using Cross Local Intrinsic Dimensionality

TL;DR

This paper introduces CrossLID, a new intrinsic dimensionality-based metric for evaluating GANs, demonstrating its effectiveness in measuring realism, detecting mode collapse, and guiding training improvements across multiple datasets.

Contribution

The paper proposes CrossLID, a novel evaluation metric based on local intrinsic dimensionality, which effectively assesses GAN quality and can enhance training processes.

Findings

CrossLID correlates strongly with GAN training progress

It detects mode collapse effectively

It is robust to noise and transformations

Abstract

Generative Adversarial Networks (GANs) are an elegant mechanism for data generation. However, a key challenge when using GANs is how to best measure their ability to generate realistic data. In this paper, we demonstrate that an intrinsic dimensional characterization of the data space learned by a GAN model leads to an effective evaluation metric for GAN quality. In particular, we propose a new evaluation measure, CrossLID, that assesses the local intrinsic dimensionality (LID) of real-world data with respect to neighborhoods found in GAN-generated samples. Intuitively, CrossLID measures the degree to which manifolds of two data distributions coincide with each other. In experiments on 4 benchmark image datasets, we compare our proposed measure to several state-of-the-art evaluation metrics. Our experiments show that CrossLID is strongly correlated with the progress of GAN training, is sensitive to mode collapse, is robust to small-scale noise and image transformations, and robust to sample size. Furthermore, we show how CrossLID can be used within the GAN training process to improve generation quality.