The Rising Entropy of English in the Attention Economy

TL;DR

This paper shows that the entropy of American English has been increasing since 1900, with higher entropy in short-form and social media, explained by an ecological model of the attention economy.

Contribution

It introduces an ecological model combining Zipf's law and information foraging to explain rising word entropy in media content over time.

Findings

Word entropy of American English has steadily increased since 1900.

Media categories differ in entropy levels, with social media highest.

The ecological model links technological changes to increased content entropy.

Abstract

We present evidence that the word entropy of American English has been rising steadily since around 1900, contrary to predictions from existing sociolinguistic theories. We also find differences in word entropy between media categories, with short-form media such as news and magazines having higher entropy than long-form media, and social media feeds having higher entropy still. To explain these results we develop an ecological model of the attention economy that combines ideas from Zipf's law and information foraging. In this model, media consumers maximize information utility rate taking into account the costs of information search, while media producers adapt to technologies that reduce search costs, driving them to generate higher entropy content in increasingly shorter formats.