Are handcrafted filters helpful for attributing AI-generated images?

TL;DR

This paper investigates the effectiveness of handcrafted filters, specifically Multi-Directional High-Pass Filters, in improving AI-generated image attribution, reducing training data needs, and enhancing model fingerprint discrimination.

Contribution

The authors introduce MHFs and a DEFL network with DMC loss, demonstrating improved attribution performance with less training data compared to existing methods.

Findings

MHFs effectively extract subtle AI image fingerprints.

The proposed method outperforms state-of-the-art in closed-set and open-set attribution.

Significant reduction in training data required for accurate attribution.

Abstract

Recently, a vast number of image generation models have been proposed, which raises concerns regarding the misuse of these artificial intelligence (AI) techniques for generating fake images. To attribute the AI-generated images, existing schemes usually design and train deep neural networks (DNNs) to learn the model fingerprints, which usually requires a large amount of data for effective learning. In this paper, we aim to answer the following two questions for AI-generated image attribution, 1) is it possible to design useful handcrafted filters to facilitate the fingerprint learning? and 2) how we could reduce the amount of training data after we incorporate the handcrafted filters? We first propose a set of Multi-Directional High-Pass Filters (MHFs) which are capable to extract the subtle fingerprints from various directions. Then, we propose a Directional Enhanced Feature Learning network (DEFL) to take both the MHFs and randomly-initialized filters into consideration. The output of the DEFL is fused with the semantic features to produce a compact fingerprint. To make the compact fingerprint discriminative among different models, we propose a Dual-Margin Contrastive (DMC) loss to tune our DEFL. Finally, we propose a reference based fingerprint classification scheme for image attribution. Experimental results demonstrate that it is indeed helpful to use our MHFs for attributing the AI-generated images. The performance of our proposed method is significantly better than the state-of-the-art for both the closed-set and open-set image attribution, where only a small amount of images are required for training.