Learning Robust Representations Of Generative Models Using Set-Based Artificial Fingerprints

TL;DR

This paper introduces a novel set-based fingerprinting method for generative models, improving source attribution and model similarity detection across various generative architectures.

Contribution

It extends fingerprinting to multiple generative model types and proposes a set-encoding contrastive training approach for more stable and accurate model attribution.

Findings

Effective fingerprinting across VAEs, Flows, GANs, and score-based models.

Enhanced stability and attribution accuracy over existing methods.

Discovery of latent model families via learned similarity metrics.

Abstract

With recent progress in deep generative models, the problem of identifying synthetic data and comparing their underlying generative processes has become an imperative task for various reasons, including fighting visual misinformation and source attribution. Existing methods often approximate the distance between the models via their sample distributions. In this paper, we approach the problem of fingerprinting generative models by learning representations that encode the residual artifacts left by the generative models as unique signals that identify the source models. We consider these unique traces (a.k.a. "artificial fingerprints") as representations of generative models, and demonstrate their usefulness in both the discriminative task of source attribution and the unsupervised task of defining a similarity between the underlying models. We first extend the existing studies on fingerprints of GANs to four representative classes of generative models (VAEs, Flows, GANs and score-based models), and demonstrate their existence and attributability. We then improve the stability and attributability of the fingerprints by proposing a new learning method based on set-encoding and contrastive training. Our set-encoder, unlike existing methods that operate on individual images, learns fingerprints from a \textit{set} of images. We demonstrate improvements in the stability and attributability through comparisons to state-of-the-art fingerprint methods and ablation studies. Further, our method employs contrastive training to learn an implicit similarity between models. We discover latent families of generative models using this metric in a standard hierarchical clustering algorithm.