BOSC: A Backdoor-based Framework for Open Set Synthetic Image Attribution

TL;DR

BOSC is a novel open-set synthetic image attribution framework that uses backdoor triggers to classify known sources and reject unknown ones, outperforming existing methods and maintaining robustness against image processing.

Contribution

The paper introduces BOSC, a backdoor-based open-set classification framework for synthetic image attribution, enabling detection of unknown sources and improving over state-of-the-art methods.

Findings

BOSC achieves superior accuracy in open-set synthetic image attribution.

The method maintains robustness against various image processing techniques.

BOSC outperforms existing approaches in experimental evaluations.

Abstract

Synthetic image attribution addresses the problem of tracing back the origin of images produced by generative models. Extensive efforts have been made to explore unique representations of generative models and use them to attribute a synthetic image to the model that produced it. Most of the methods classify the models or the architectures among those in a closed set without considering the possibility that the system is fed with samples produced by unknown architectures. With the continuous progress of AI technology, new generative architectures continuously appear, thus driving the attention of researchers towards the development of tools capable of working in open-set scenarios. In this paper, we propose a framework for open set attribution of synthetic images, named BOSC (Backdoor-based Open Set Classification), that relies on the concept of backdoor attacks to design a classifier with rejection option. BOSC works by purposely injecting class-specific triggers inside a portion of the images in the training set to induce the network to establish a matching between class features and trigger features. The behavior of the trained model with respect to triggered samples is then exploited at test time to perform sample rejection using an ad-hoc score. Experiments show that the proposed method has good performance, always surpassing the state-of-the-art. Robustness against image processing is also very good. Although we designed our method for the task of synthetic image attribution, the proposed framework is a general one and can be used for other image forensic applications.