Have LLMs Reopened the Pandora's Box of AI-Generated Fake News?

TL;DR

This study investigates the use of large language models in creating and detecting fake news, revealing that LLMs outperform humans in real news detection and are comparable in fake news detection, highlighting new challenges in AI-generated misinformation.

Contribution

The paper presents empirical findings from a large-scale competition on AI-generated fake news creation and detection, analyzing human and AI performance and strategies to improve fake news credibility.

Findings

LLMs are ~68% more effective at detecting real news than humans

Performance of LLMs and humans in fake news detection is comparable (~60%)

Visual elements impact fake news detection accuracy

Abstract

With the rise of AI-generated content spewed at scale from large language models (LLMs), genuine concerns about the spread of fake news have intensified. The perceived ability of LLMs to produce convincing fake news at scale poses new challenges for both human and automated fake news detection systems. To address this gap, this paper presents the findings from a university-level competition that aimed to explore how LLMs can be used by humans to create fake news, and to assess the ability of human annotators and AI models to detect it. A total of 110 participants used LLMs to create 252 unique fake news stories, and 84 annotators participated in the detection tasks. Our findings indicate that LLMs are ~68% more effective at detecting real news than humans. However, for fake news detection, the performance of LLMs and humans remains comparable (~60% accuracy). Additionally, we examine the impact of visual elements (e.g., pictures) in news on the accuracy of detecting fake news stories. Finally, we also examine various strategies used by fake news creators to enhance the credibility of their AI-generated content. This work highlights the increasing complexity of detecting AI-generated fake news, particularly in collaborative human-AI settings.