Adversarial Robustness of AI-Generated Image Detectors in the Real World

TL;DR

This paper investigates the vulnerability of AI-generated image detectors to adversarial attacks in real-world scenarios, revealing significant robustness issues and suggesting the need for improved detection methods.

Contribution

It demonstrates that current state-of-the-art detectors are highly vulnerable to adversarial examples, even in black-box settings and under image degradation, highlighting critical robustness challenges.

Findings

Adversarial attacks significantly reduce detection accuracy.

Most attacks remain effective despite image degradation.

Using robust pre-trained features improves but does not fully solve robustness issues.

Abstract

The rapid advancement of Generative Artificial Intelligence (GenAI) capabilities is accompanied by a concerning rise in its misuse. In particular the generation of credible misinformation in the form of images poses a significant threat to the public trust in democratic processes. Consequently, there is an urgent need to develop tools to reliably distinguish between authentic and AI-generated content. The majority of detection methods are based on neural networks that are trained to recognize forensic artifacts. In this work, we demonstrate that current state-of-the-art classifiers are vulnerable to adversarial examples under real-world conditions. Through extensive experiments, comprising four detection methods and five attack algorithms, we show that an attacker can dramatically decrease classification performance, without internal knowledge of the detector's architecture. Notably, most attacks remain effective even when images are degraded during the upload to, e.g., social media platforms. In a case study, we demonstrate that these robustness challenges are also found in commercial tools by conducting black-box attacks on HIVE, a proprietary online GenAI media detector. In addition, we evaluate the robustness of using generated features of a robust pre-trained model and showed that this increases the robustness, while not reaching the performance on benign inputs. These results, along with the increasing potential of GenAI to erode public trust, underscore the need for more research and new perspectives on methods to prevent its misuse.