So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection

TL;DR

This paper introduces So-Fake, a large-scale social media image forgery dataset, a challenging out-of-domain benchmark, and an advanced detection framework that significantly improves accuracy and localization, advancing research in AI-generated image detection.

Contribution

The paper presents a comprehensive social media-oriented forgery dataset, a large out-of-domain benchmark, and a reinforcement learning-based detection framework, addressing current limitations in diversity, scale, and generalization.

Findings

So-Fake-Set contains over 2 million images from 35 generative models.

So-Fake-OOD provides a challenging out-of-domain test for real-world scenarios.

So-Fake-R1 outperforms previous methods with a 1.3% higher detection accuracy and 4.5% better localization IoU.

Abstract

Recent advances in AI-powered generative models have enabled the creation of increasingly realistic synthetic images, posing significant risks to information integrity and public trust on social media platforms. While robust detection frameworks and diverse, large-scale datasets are essential to mitigate these risks, existing academic efforts remain limited in scope: current datasets lack the diversity, scale, and realism required for social media contexts, while detection methods struggle with generalization to unseen generative technologies. To bridge this gap, we introduce So-Fake-Set, a comprehensive social media-oriented dataset with over 2 million high-quality images, diverse generative sources, and photorealistic imagery synthesized using 35 state-of-the-art generative models. To rigorously evaluate cross-domain robustness, we establish a novel and large-scale (100K) out-of-domain benchmark (So-Fake-OOD) featuring synthetic imagery from commercial models explicitly excluded from the training distribution, creating a realistic testbed for evaluating real-world performance. Leveraging these resources, we present So-Fake-R1, an advanced vision-language framework that employs reinforcement learning for highly accurate forgery detection, precise localization, and explainable inference through interpretable visual rationales. Extensive experiments show that So-Fake-R1 outperforms the second-best method, with a 1.3% gain in detection accuracy and a 4.5% increase in localization IoU. By integrating a scalable dataset, a challenging OOD benchmark, and an advanced detection framework, this work establishes a new foundation for social media-centric forgery detection research. The code, models, and datasets will be released publicly.