Testing the Predictions of Surprisal Theory in 11 Languages

TL;DR

This study provides crosslinguistic evidence supporting Surprisal Theory by analyzing reading times and predictability in eleven diverse languages, confirming key theoretical predictions across language families.

Contribution

It is the first comprehensive multilingual analysis validating Surprisal Theory's predictions on reading times across diverse languages.

Findings

Surprisal predicts reading times in all examined languages.

Expected surprisal (entropy) also predicts reading times crosslinguistically.

The link between surprisal and reading times is linear across languages.

Abstract

A fundamental result in psycholinguistics is that less predictable words take a longer time to process. One theoretical explanation for this finding is Surprisal Theory (Hale, 2001; Levy, 2008), which quantifies a word's predictability as its surprisal, i.e. its negative log-probability given a context. While evidence supporting the predictions of Surprisal Theory have been replicated widely, most have focused on a very narrow slice of data: native English speakers reading English texts. Indeed, no comprehensive multilingual analysis exists. We address this gap in the current literature by investigating the relationship between surprisal and reading times in eleven different languages, distributed across five language families. Deriving estimates from language models trained on monolingual and multilingual corpora, we test three predictions associated with surprisal theory: (i) whether surprisal is predictive of reading times; (ii) whether expected surprisal, i.e. contextual entropy, is predictive of reading times; (iii) and whether the linking function between surprisal and reading times is linear. We find that all three predictions are borne out crosslinguistically. By focusing on a more diverse set of languages, we argue that these results offer the most robust link to-date between information theory and incremental language processing across languages.