Integrating large language models and active inference to understand eye movements in reading and dyslexia

TL;DR

This paper introduces a hierarchical active inference model integrating large language models to simulate reading and eye movements, providing insights into normal reading and dyslexia by linking predictive processing to eye movement patterns.

Contribution

The novel model combines large language models with active inference to simulate reading and eye movements, advancing understanding of cognitive processes and dyslexia.

Findings

Model accurately simulates reading of known and unknown words

Attenuating priors results in fragmented reading similar to dyslexia

Model supports dual route theories of reading

Abstract

We present a novel computational model employing hierarchical active inference to simulate reading and eye movements. The model characterizes linguistic processing as inference over a hierarchical generative model, facilitating predictions and inferences at various levels of granularity, from syllables to sentences. Our approach combines the strengths of large language models for realistic textual predictions and active inference for guiding eye movements to informative textual information, enabling the testing of predictions. The model exhibits proficiency in reading both known and unknown words and sentences, adhering to the distinction between lexical and nonlexical routes in dual route theories of reading. Our model therefore provides a novel approach to understand the cognitive processes underlying reading and eye movements, within a predictive processing framework. Furthermore, our model can potentially aid in understanding how maladaptive predictive processing can produce reading deficits associated with dyslexia. As a proof of concept, we show that attenuating the contribution of priors during the reading process leads to incorrect inferences and a more fragmented reading style, characterized by a greater number of shorter saccades, aligning with empirical findings regarding eye movements in dyslexic individuals. In summary, our model represents a significant advancement in comprehending the cognitive processes involved in reading and eye movements, with potential implications for understanding dyslexia in terms of maladaptive inference.