GMValuator: Similarity-based Data Valuation for Generative Models

TL;DR

GMValuator is a novel, training-free, and model-agnostic method for data valuation in generative models, using similarity matching and image quality assessment to efficiently evaluate training data contributions.

Contribution

It introduces GMValuator, the first approach for data valuation in generative models that is training-free, model-agnostic, and incorporates similarity matching and image quality metrics.

Findings

Effective on benchmark datasets

Works across various generative architectures

Outperforms existing methods in robustness and efficiency

Abstract

Data valuation plays a crucial role in machine learning. Existing data valuation methods, mainly focused on discriminative models, overlook generative models that have gained attention recently. In generative models, data valuation measures the impact of training data on generated datasets. Very few existing attempts at data valuation methods designed for deep generative models either concentrate on specific models or lack robustness in their outcomes. Moreover, efficiency still reveals vulnerable shortcomings. We formulate the data valuation problem in generative models from a similarity matching perspective to bridge the gaps. Specifically, we introduce Generative Model Valuator (GMValuator), the first training-free and model-agnostic approach to providing data valuation for image generation tasks. It empowers efficient data valuation through our innovative similarity matching module, calibrates biased contributions by incorporating image quality assessment, and attributes credits to all training samples based on their contributions to the generated samples. Additionally, we introduce four evaluation criteria for assessing data valuation methods in generative models. GMValuator is extensively evaluated on benchmark and high-resolution datasets and various mainstream generative architectures to demonstrate its effectiveness.