LLM as a Broken Telephone: Iterative Generation Distorts Information

TL;DR

This paper investigates how iterative use of large language models can distort information over multiple generations, highlighting the effects of language choice and chain complexity, and proposing strategies to mitigate degradation.

Contribution

It introduces an experimental framework to analyze information distortion in LLMs during iterative generation and offers insights into mitigation techniques.

Findings

Distortion accumulates over multiple generations.

Language choice influences the degree of distortion.

Strategic prompting can reduce information degradation.

Abstract

As large language models are increasingly responsible for online content, concerns arise about the impact of repeatedly processing their own outputs. Inspired by the "broken telephone" effect in chained human communication, this study investigates whether LLMs similarly distort information through iterative generation. Through translation-based experiments, we find that distortion accumulates over time, influenced by language choice and chain complexity. While degradation is inevitable, it can be mitigated through strategic prompting techniques. These findings contribute to discussions on the long-term effects of AI-mediated information propagation, raising important questions about the reliability of LLM-generated content in iterative workflows.