Detecting GAN generated errors

TL;DR

This paper introduces a novel pixel-level error detection method for GAN-generated images, enabling better quality assessment and understanding of generated samples by identifying specific error regions.

Contribution

It proposes a new approach that detects errors within generated images using pixel-wise analysis and attention mechanisms, improving quality evaluation beyond traditional metrics.

Findings

Error detection correlates with FID scores

Method identifies localized errors in images

Enhances understanding of GAN sample quality

Abstract

Despite an impressive performance from the latest GAN for generating hyper-realistic images, GAN discriminators have difficulty evaluating the quality of an individual generated sample. This is because the task of evaluating the quality of a generated image differs from deciding if an image is real or fake. A generated image could be perfect except in a single area but still be detected as fake. Instead, we propose a novel approach for detecting where errors occur within a generated image. By collaging real images with generated images, we compute for each pixel, whether it belongs to the real distribution or generated distribution. Furthermore, we leverage attention to model long-range dependency; this allows detection of errors which are reasonable locally but not holistically. For evaluation, we show that our error detection can act as a quality metric for an individual image, unlike FID and IS. We leverage Improved Wasserstein, BigGAN, and StyleGAN to show a ranking based on our metric correlates impressively with FID scores. Our work opens the door for better understanding of GAN and the ability to select the best samples from a GAN model.