When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text

TL;DR

This paper investigates the challenge of detecting AI-generated social media posts, revealing that fine-tuned models can produce human-like content that evades detection, especially when the attacker keeps their models private.

Contribution

The study creates a large dataset of AI-generated social media posts and demonstrates the significant decline in detectability when models are fine-tuned and kept private.

Findings

Detection accuracy drops when models are fine-tuned and private.

Humans struggle to distinguish AI-generated posts from real ones.

Detection algorithms are vulnerable to fine-tuned LLMs.

Abstract

Detecting AI-generated text is a difficult problem to begin with; detecting AI-generated text on social media is made even more difficult due to the short text length and informal, idiosyncratic language of the internet. It is nonetheless important to tackle this problem, as social media represents a significant attack vector in online influence campaigns, which may be bolstered through the use of mass-produced AI-generated posts supporting (or opposing) particular policies, decisions, or events. We approach this problem with the mindset and resources of a reasonably sophisticated threat actor, and create a dataset of 505,159 AI-generated social media posts from a combination of open-source, closed-source, and fine-tuned LLMs, covering 11 different controversial topics. We show that while the posts can be detected under typical research assumptions about knowledge of and access to the generating models, under the more realistic assumption that an attacker will not release their fine-tuned model to the public, detectability drops dramatically. This result is confirmed with a human study. Ablation experiments highlight the vulnerability of various detection algorithms to fine-tuned LLMs. This result has implications across all detection domains, since fine-tuning is a generally applicable and realistic LLM use case.