Passing the Turing Test in Political Discourse: Fine-Tuning LLMs to Mimic Polarized Social Media Comments

TL;DR

This paper investigates how fine-tuned large language models can generate highly plausible, polarized political comments on social media, raising ethical concerns and highlighting the need for detection tools and regulation.

Contribution

It demonstrates that fine-tuned LLMs can mimic polarized discourse convincingly, revealing risks of AI-driven manipulation in political online environments.

Findings

Fine-tuned LLMs produce human-like, provocative comments.

Models trained on partisan data amplify ideological bias.

Generated content is often indistinguishable from human comments.

Abstract

The increasing sophistication of large language models (LLMs) has sparked growing concerns regarding their potential role in exacerbating ideological polarization through the automated generation of persuasive and biased content. This study explores the extent to which fine-tuned LLMs can replicate and amplify polarizing discourse within online environments. Using a curated dataset of politically charged discussions extracted from Reddit, we fine-tune an open-source LLM to produce context-aware and ideologically aligned responses. The model's outputs are evaluated through linguistic analysis, sentiment scoring, and human annotation, with particular attention to credibility and rhetorical alignment with the original discourse. The results indicate that, when trained on partisan data, LLMs are capable of producing highly plausible and provocative comments, often indistinguishable from those written by humans. These findings raise significant ethical questions about the use of AI in political discourse, disinformation, and manipulation campaigns. The paper concludes with a discussion of the broader implications for AI governance, platform regulation, and the development of detection tools to mitigate adversarial fine-tuning risks.