Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics?

TL;DR

This study evaluates whether state-of-the-art LLMs can convincingly mimic the writing styles of famous literary and political figures, finding current models are highly detectable and lack nuanced stylistic variability.

Contribution

It introduces a comprehensive framework combining linguistic, statistical, and machine learning methods to benchmark LLMs' ability to emulate human authorship styles.

Findings

AI-generated texts are highly distinguishable from human writing using stylometric features.

Perplexity is the most significant feature for differentiating AI and human texts.

LLMs do not fully capture the stylistic variance and affective density of human authors.

Abstract

Amidst the rising capabilities of generative AI to mimic specific human styles, this study investigates the ability of state-of-the-art large language models (LLMs), including GPT-4o, Gemini 1.5 Pro, and Claude Sonnet 3.5, to emulate the authorial signatures of prominent literary and political figures: Walt Whitman, William Wordsworth, Donald Trump, and Barack Obama. Utilizing a zero-shot prompting framework with strict thematic alignment, we generated synthetic corpora evaluated through a complementary framework combining transformer-based classification (BERT) and interpretable machine learning (XGBoost). Our methodology integrates Linguistic Inquiry and Word Count (LIWC) markers, perplexity, and readability indices to assess the divergence between AI-generated and human-authored text. Results demonstrate that AI-generated mimicry remains highly detectable, with XGBoost models trained on a restricted set of eight stylometric features achieving accuracy comparable to high-dimensional neural classifiers. Feature importance analyses identify perplexity as the primary discriminative metric, revealing a significant divergence in the stochastic regularity of AI outputs compared to the higher variability of human writing. While LLMs exhibit distributional convergence with human authors on low-dimensional heuristic features, such as syntactic complexity and readability, they do not yet fully replicate the nuanced affective density and stylistic variance inherent in the human-authored corpus. By isolating the specific statistical gaps in current generative mimicry, this study provides a comprehensive benchmark for LLM stylistic behavior and offers critical insights for authorship attribution in the digital humanities and social media.