The Deployment Gap in AI Media Detection: Platform-Aware and Visually Constrained Adversarial Evaluation

TL;DR

This paper introduces a platform-aware adversarial evaluation framework revealing significant robustness gaps in AI media detectors under realistic deployment transformations and visual constraints.

Contribution

It presents a new evaluation framework modeling deployment transforms and visual constraints, exposing vulnerabilities of detectors previously considered robust.

Findings

Detectors' AUC drops significantly under realistic transforms.

Universal perturbations exist across inputs under visual constraints.

Calibration collapse occurs under attack, leading to overconfidence in errors.

Abstract

Recent AI media detectors report near-perfect performance under clean laboratory evaluation, yet their robustness under realistic deployment conditions remains underexplored. In practice, AI-generated images are resized, compressed, re-encoded, and visually modified before being shared on online platforms. We argue that this creates a deployment gap between laboratory robustness and real-world reliability. In this work, we introduce a platform-aware adversarial evaluation framework for AI media detection that explicitly models deployment transforms (e.g., resizing, compression, screenshot-style distortions) and constrains perturbations to visually plausible meme-style bands rather than full-image noise. Under this threat model, detectors achieving AUC $\approx$ 0{.}99 in clean settings experience substantial degradation. Per-image platform-aware attacks reduce AUC to significantly lower levels and achieve high fake-to-real misclassification rates, despite strict visual constraints. We further demonstrate that universal perturbations exist even under localized band constraints, revealing shared vulnerability directions across inputs. Beyond accuracy degradation, we observe pronounced calibration collapse under attack, where detectors become confidently incorrect. Our findings highlight that robustness measured under clean conditions substantially overestimates deployment robustness. We advocate for platform-aware evaluation as a necessary component of future AI media security benchmarks and release our evaluation framework to facilitate standardized robustness assessment.