Unsupervised evaluation of GAN sample quality: Introducing the TTJac Score

TL;DR

The paper introduces the TTJac score, a novel, efficient, data-free metric for evaluating GAN-generated image quality, improving fidelity assessment without high memory costs.

Contribution

It presents the TTJac score, a new theoretical and practical metric for assessing GAN sample quality that is computationally efficient and enhances fidelity-variability trade-offs.

Findings

TTJac score effectively evaluates GAN sample fidelity.

The metric improves the fidelity-variability trade-off.

Experimental validation on multiple datasets confirms its utility.

Abstract

Evaluation metrics are essential for assessing the performance of generative models in image synthesis. However, existing metrics often involve high memory and time consumption as they compute the distance between generated samples and real data points. In our study, the new evaluation metric called the "TTJac score" is proposed to measure the fidelity of individual synthesized images in a data-free manner. The study first establishes a theoretical approach to directly evaluate the generated sample density. Then, a method incorporating feature extractors and discrete function approximation through tensor train is introduced to effectively assess the quality of generated samples. Furthermore, the study demonstrates that this new metric can be used to improve the fidelity-variability trade-off when applying the truncation trick. The experimental results of applying the proposed metric to StyleGAN 2 and StyleGAN 2 ADA models on FFHQ, AFHQ-Wild, LSUN-Cars, and LSUN-Horse datasets are presented. The code used in this research will be made publicly available online for the research community to access and utilize.