Investigating the Timescales of Language Processing with EEG and Language Models

TL;DR

This paper investigates the temporal dynamics of language processing by aligning EEG data with representations from a transformer-based language model, revealing how different model layers relate to neural responses during comprehension.

Contribution

It introduces a method to analyze the interaction between neural activity and model representations across layers using TRF and LDA, providing new insights into language processing mechanisms.

Findings

Distinct TRF patterns across model layers highlight different contributions to lexical and syntactic processing.

LDA isolates POS representations, showing their influence on neural responses.

EEG effectively probes the timing of language processing in the brain.

Abstract

This study explores the temporal dynamics of language processing by examining the alignment between word representations from a pre-trained transformer-based language model, and EEG data. Using a Temporal Response Function (TRF) model, we investigate how neural activity corresponds to model representations across different layers, revealing insights into the interaction between artificial language models and brain responses during language comprehension. Our analysis reveals patterns in TRFs from distinct layers, highlighting varying contributions to lexical and compositional processing. Additionally, we used linear discriminant analysis (LDA) to isolate part-of-speech (POS) representations, offering insights into their influence on neural responses and the underlying mechanisms of syntactic processing. These findings underscore EEG's utility for probing language processing dynamics with high temporal resolution. By bridging artificial language models and neural activity, this study advances our understanding of their interaction at fine timescales.