How LLMs Distort Our Written Language
Marwa Abdulhai, Isadora White, Yanming Wan, Ibrahim Qureshi, Joel Leibo, Max Kleiman-Weiner, Natasha Jaques

TL;DR
This paper investigates how large language models (LLMs) influence human writing, revealing they often alter intended meaning, tone, and creativity, with significant implications for culture and science.
Contribution
It provides empirical evidence that LLMs change the semantic content and evaluation of human writing, highlighting potential risks of widespread AI-assisted writing.
Findings
LLMs increase neutral responses in essays by 70%
LLMs significantly alter the semantic meaning even with minimal edits
AI-generated peer reviews tend to favor scores and de-emphasize clarity
Abstract
Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing, but also consistently alter the intended meaning. First, we conduct a human user study to understand how people actually interact with LLMs when using them for writing. Our findings reveal that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice. Next, using a dataset of human-written essays that was collected in 2021 before the widespread release of LLMs, we study how asking an LLM to revise the essay based on the human-written feedback in the dataset induces large changes in the resulting content and meaning. We find that even…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Machine Learning in Materials Science · Computational and Text Analysis Methods
