Assessing Sample Quality via the Latent Space of Generative Models

TL;DR

This paper introduces a novel method for assessing the quality of generated samples by analyzing the latent space density of the generative model itself, offering a domain-agnostic and computationally efficient alternative to existing techniques.

Contribution

The authors propose a latent density score for sample quality assessment that is applicable across various models and domains, overcoming limitations of feature-based methods.

Findings

High correlation between latent density score and sample quality.

Method reduces computational cost compared to previous approaches.

Improves downstream tasks like few-shot classification and image editing.

Abstract

Advances in generative models increase the need for sample quality assessment. To do so, previous methods rely on a pre-trained feature extractor to embed the generated samples and real samples into a common space for comparison. However, different feature extractors might lead to inconsistent assessment outcomes. Moreover, these methods are not applicable for domains where a robust, universal feature extractor does not yet exist, such as medical images or 3D assets. In this paper, we propose to directly examine the latent space of the trained generative model to infer generated sample quality. This is feasible because the quality a generated sample directly relates to the amount of training data resembling it, and we can infer this information by examining the density of the latent space. Accordingly, we use a latent density score function to quantify sample quality. We show that the proposed score correlates highly with the sample quality for various generative models including VAEs, GANs and Latent Diffusion Models. Compared with previous quality assessment methods, our method has the following advantages: 1) pre-generation quality estimation with reduced computational cost, 2) generalizability to various domains and modalities, and 3) applicability to latent-based image editing and generation methods. Extensive experiments demonstrate that our proposed methods can benefit downstream tasks such as few-shot image classification and latent face image editing. Code is available at https://github.com/cvlab-stonybrook/LS-sample-quality.