Pros and Cons of GAN Evaluation Measures: New Developments

TL;DR

This paper reviews recent developments in GAN evaluation metrics, highlighting new dimensions like bias and fairness, and discusses their relevance to issues like deepfakes, emphasizing ongoing challenges and improvements.

Contribution

It updates previous work by introducing new evaluation dimensions and connecting GAN assessment to societal concerns such as bias, fairness, and deepfake detection.

Findings

New evaluation dimensions like bias and fairness are gaining importance.

GAN evaluation remains an open challenge with room for improvement.

Connections between GAN metrics and deepfake issues are discussed.

Abstract

This work is an update of a previous paper on the same topic published a few years ago. With the dramatic progress in generative modeling, a suite of new quantitative and qualitative techniques to evaluate models has emerged. Although some measures such as Inception Score, Frechet Inception Distance, Precision-Recall, and Perceptual Path Length are relatively more popular, GAN evaluation is not a settled issue and there is still room for improvement. Here, I describe new dimensions that are becoming important in assessing models (e.g. bias and fairness) and discuss the connection between GAN evaluation and deepfakes. These are important areas of concern in the machine learning community today and progress in GAN evaluation can help mitigate them.