GPT as ghostwriter at the White House

TL;DR

This study analyzes ChatGPT 3.5's writing style by comparing its generated US presidential addresses with actual speeches, revealing stylistic differences and limitations in style imitation.

Contribution

It provides a detailed comparison of ChatGPT's generated speeches with real presidential addresses, highlighting stylistic patterns and the model's inability to fully mimic specific authors.

Findings

ChatGPT overuses 'we', nouns, and commas.

Generated speeches have longer sentences and fewer verbs.

ChatGPT maintains a neutral, positive tone with symbolic language.

Abstract

Recently several large language models (LLMs) have demonstrated their capability to generate a message in response to a user request. Such scientific breakthroughs promote new perspectives but also some fears. The main focus of this study is to analyze the written style of one LLM called ChatGPT 3.5 by comparing its generated messages with those of the recent US presidents. To achieve this objective, we compare the State of the Union addresses written by Reagan to Obama with those automatically produced by ChatGPT. We found that ChatGPT tends to overuse the lemma "we" as well as nouns and commas. On the other hand, the generated speeches employ less verbs and include, in mean, longer sentences. Even when imposing a given style to ChatGPT, the resulting speech remains distinct from messages written by the target author. Moreover, ChatGPT opts for a neutral tone with mainly positive emotional expressions and symbolic terms (e.g., freedom, nation). Finally, we show that the GPT's style exposes distinct features compared to real presidential addresses.