Unsupervised Synthetic Image Attribution: Alignment and Disentanglement

TL;DR

This paper introduces an unsupervised method for synthetic image attribution that leverages contrastive self-supervised learning and representation disentanglement, outperforming supervised methods on real-world benchmarks.

Contribution

It presents a novel unsupervised approach combining alignment and disentanglement, with a theoretical explanation linking contrastive learning to concept matching.

Findings

Outperforms supervised methods on AbC benchmarks

Utilizes contrastive self-supervised models for alignment

Provides a theoretical framework for unsupervised attribution

Abstract

As the quality of synthetic images improves, identifying the underlying concepts of model-generated images is becoming increasingly crucial for copyright protection and ensuring model transparency. Existing methods achieve this attribution goal by training models using annotated pairs of synthetic images and their original training sources. However, obtaining such paired supervision is challenging, as it requires either well-designed synthetic concepts or precise annotations from millions of training sources. To eliminate the need for costly paired annotations, in this paper, we explore the possibility of unsupervised synthetic image attribution. We propose a simple yet effective unsupervised method called Alignment and Disentanglement. Specifically, we begin by performing basic concept alignment using contrastive self-supervised learning. Next, we enhance the model's attribution ability by promoting representation disentanglement with the Infomax loss. This approach is motivated by an interesting observation: contrastive self-supervised models, such as MoCo and DINO, inherently exhibit the ability to perform simple cross-domain alignment. By formulating this observation as a theoretical assumption on cross-covariance, we provide a theoretical explanation of how alignment and disentanglement can approximate the concept-matching process through a decomposition of the canonical correlation analysis objective. On the real-world benchmarks, AbC, we show that our unsupervised method surprisingly outperforms the supervised methods. As a starting point, we expect our intuitive insights and experimental findings to provide a fresh perspective on this challenging task.