An information-theoretic model of shallow and deep language comprehension

TL;DR

This paper introduces an information-theoretic model of language comprehension that balances accuracy and processing effort, linking it to neural signals and validating it with experimental data.

Contribution

It formalizes the trade-off between comprehension depth and accuracy under resource constraints using information theory, connecting it to neural and behavioral data.

Findings

Model explains EEG and reading time data

Quantifies shallow to deep processing progression

Validates with large-scale experimental datasets

Abstract

A large body of work in psycholinguistics has focused on the idea that online language comprehension can be shallow or `good enough': given constraints on time or available computation, comprehenders may form interpretations of their input that are plausible but inaccurate. However, this idea has not yet been linked with formal theories of computation under resource constraints. Here we use information theory to formulate a model of language comprehension as an optimal trade-off between accuracy and processing depth, formalized as bits of information extracted from the input, which increases with processing time. The model provides a measure of processing effort as the change in processing depth, which we link to EEG signals and reading times. We validate our theory against a large-scale dataset of garden path sentence reading times, and EEG experiments featuring N400, P600 and biphasic ERP effects. By quantifying the timecourse of language processing as it proceeds from shallow to deep, our model provides a unified framework to explain behavioral and neural signatures of language comprehension.