Who Said That? Benchmarking Social Media AI Detection
Wanyun Cui, Linqiu Zhang, Qianle Wang, Shuyang Cai
TL;DR
This paper introduces SAID, a new benchmark for evaluating AI-generated text detection on real social media platforms, revealing high human accuracy and emphasizing the challenge of practical detection in online environments.
Contribution
The paper presents SAID, a realistic benchmark using real social media data, and explores human and user-oriented detection methods, highlighting challenges and improvements in AI-text detection.
Findings
Humans can distinguish AI from human text with 96.5% accuracy.
Detection is more challenging on real social media data than simulated datasets.
User-oriented detection methods improve accuracy significantly.
Abstract
AI-generated text has proliferated across various online platforms, offering both transformative prospects and posing significant risks related to misinformation and manipulation. Addressing these challenges, this paper introduces SAID (Social media AI Detection), a novel benchmark developed to assess AI-text detection models' capabilities in real social media platforms. It incorporates real AI-generate text from popular social media platforms like Zhihu and Quora. Unlike existing benchmarks, SAID deals with content that reflects the sophisticated strategies employed by real AI users on the Internet which may evade detection or gain visibility, providing a more realistic and challenging evaluation landscape. A notable finding of our study, based on the Zhihu dataset, reveals that annotators can distinguish between AI-generated and human-generated texts with an average accuracy rate of…
Peer Reviews
Decision·ICLR 2024 Conference Withdrawn Submission
- The number of collected comments is large - The methods are evaluated on two languages - Human verification is also used - Improvement of the classification is achieved
Unfortunately, I see a major issue with the data collection. 1. The classification of AI content is mainly based on the pre-labelled data from one of the websites, which means, that their classifier is defined as ground truth. We do not know anything about the used classifier. It could be one of the three classifiers used in the paper, it could also be one specifically trained on the content of the webiste 2. If a user was labelled as AI content for three of their posts, all content was labell
1. This research problem is interesting 2. The writing of this paper is easy to follow
My main concerns are on the data collection method of this paper. I am not convinced by the experiments. The reasons are as follows: 1. The reliability of the original label pool: The initial label pool is from the API of Quora and Zhihu. But their detection ability is not supposed to be better than the SOTA detectors and human verifiers, as they do not have more knowledge about this task. Actually, it is very possible that their initial labels are just from SOTA detectors and human verifiers.
The paper focuses on an important and timely issue that I believe will become even more important in the future, so these research efforts and benchmarks can equip the community with the necessary tools to being able to identify machine-generated text. An important strength of the paper is that it releases a large-scale dataset of AI and human-generated posts shared on two social media platforms across two languages (English and Chinese). The multilingual and multi-platform aspect of the release
I think that the paper has some considerable flaws regarding the methods/assumptions made by the paper, the interpretation of the results, the lack of validation of the employed methods, and the lack of scope. I elaborate on my main concerns along with some suggestions on how to improve the manuscript below. **Validation and Assumptions:** I think that the paper uses methods for identifying machine-generated text that are not properly validated. Specifically, the paper uses labels provided by Z
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMisinformation and Its Impacts · Topic Modeling · Computational and Text Analysis Methods