The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter

TL;DR

This study investigates how the specific wording of messages on Twitter influences their retweet count, controlling for author and topic effects, and finds that wording does have a measurable impact on message propagation.

Contribution

The paper introduces a natural experiment approach to isolate wording effects on message spread, demonstrating that computational methods can outperform humans in predicting retweet success.

Findings

Humans outperform chance in predicting which tweet version gets more retweets.

Computational models outperform both humans and baseline methods.

Wording has a significant, measurable effect on message propagation.

Abstract

Consider a person trying to spread an important message on a social network. He/she can spend hours trying to craft the message. Does it actually matter? While there has been extensive prior work looking into predicting popularity of social-media content, the effect of wording per se has rarely been studied since it is often confounded with the popularity of the author and the topic. To control for these confounding factors, we take advantage of the surprising fact that there are many pairs of tweets containing the same url and written by the same user but employing different wording. Given such pairs, we ask: which version attracts more retweets? This turns out to be a more difficult task than predicting popular topics. Still, humans can answer this question better than chance (but far from perfectly), and the computational methods we develop can do better than both an average human and a strong competing method trained on non-controlled data.