An Unsupervised Way to Understand Artifact Generating Internal Units in Generative Neural Networks

TL;DR

This paper introduces an unsupervised metric based on local activation to detect and correct artifacts in GAN-generated images, improving evaluation of individual sample quality without extra supervision.

Contribution

It proposes the concept of local activation and a new metric to identify artifact-generating units in GANs without additional supervision or external networks.

Findings

Effectively detects artifact generations across datasets

Can be used to correct artifacts in generated images

Provides a geometrical insight into low fidelity issues

Abstract

Despite significant improvements on the image generation performance of Generative Adversarial Networks (GANs), generations with low visual fidelity still have been observed. As widely used metrics for GANs focus more on the overall performance of the model, evaluation on the quality of individual generations or detection of defective generations is challenging. While recent studies try to detect featuremap units that cause artifacts and evaluate individual samples, these approaches require additional resources such as external networks or a number of training data to approximate the real data manifold. In this work, we propose the concept of local activation, and devise a metric on the local activation to detect artifact generations without additional supervision. We empirically verify that our approach can detect and correct artifact generations from GANs with various datasets. Finally, we discuss a geometrical analysis to partially reveal the relation between the proposed concept and low visual fidelity.