FLD+: Data-efficient Evaluation Metric for Generative Models

TL;DR

FLD+ is a new, data-efficient, and adaptable image quality metric based on normalizing flows, outperforming FID in reliability, stability, and domain flexibility, especially with limited data.

Contribution

Introduces FLD+, a flow-based likelihood distance metric that is more reliable, data-efficient, and adaptable to new domains than existing metrics like FID.

Findings

FLD+ exhibits strongly monotonic behavior with image degradations.

FLD+ requires two orders of magnitude fewer images than FID for stable results.

FLD+ can be retrained on new domains such as medical images.

Abstract

We introduce a new metric to assess the quality of generated images that is more reliable, data-efficient, compute-efficient, and adaptable to new domains than the previous metrics, such as Fr\'echet Inception Distance (FID). The proposed metric is based on normalizing flows, which allows for the computation of density (exact log-likelihood) of images from any domain. Thus, unlike FID, the proposed Flow-based Likelihood Distance Plus (FLD+) metric exhibits strongly monotonic behavior with respect to different types of image degradations, including noise, occlusion, diffusion steps, and generative model size. Additionally, because normalizing flow can be trained stably and efficiently, FLD+ achieves stable results with two orders of magnitude fewer images than FID (which requires more images to reliably compute Fr\'echet distance between features of large samples of real and generated images). We made FLD+ computationally even more efficient by applying normalizing flows to features extracted in a lower-dimensional latent space instead of using a pre-trained network. We also show that FLD+ can easily be retrained on new domains, such as medical images, unlike the networks behind previous metrics -- such as InceptionNetV3 pre-trained on ImageNet.