# Prediction, syntax and semantic grounding in the brain and large language models

## TL;DR

The study explores how the brain processes language by combining brain imaging with a large language model to understand how syntax and meaning are anticipated during natural speech.

## Contribution

The novel contribution is using combined EEG-MEG and a transformer-based language model to investigate word-class-specific neural responses during naturalistic language comprehension.

## Key findings

- Neural responses for different word classes showed reproducible spatio-temporal patterns with pre-onset activity for nouns.
- Nouns showed enhanced anticipatory processing and deeper semantic grounding in sensorimotor cortices compared to verbs.
- The transformer-based model Llama provided a computational reference frame that complements neural findings at the word-class level.

## Abstract

Language comprehension involves continuous anticipation of upcoming linguistic input, requiring the rapid integration of syntactic structure and semantic information. To capture the spatio-temporal dynamics of such anticipatory processes during naturalistic language comprehension, we combined electroencephalography (EEG) and magnetoencephalography (MEG), leveraging their complementary sensitivities and high temporal resolution. Using this combined EEG-MEG approach, we investigated word-class-specific neural responses during continuous speech perception and related these findings to word class-level predictability and representational structure in a large language model. Twenty-nine healthy participants listened to a German audio book while their neural responses were recorded. Event-related fields and event-related potentials for different word classes showed highly reproducible, characteristic spatio-temporal signatures, including significant pre-onset activity for nouns, suggesting enhanced anticipatory processing of this word class. Source-space analyses revealed activity patterns extending beyond temporal regions into areas compatible with sensorimotor cortices, suggesting a deeper semantic grounding of nouns in e.g. sensory experiences than verbs. By analyzing word class-specific predictability and representational structure in the transformer-based language model Llama, we provide a computational reference frame that complements the neural findings at the level of word classes. These findings highlight the power of simultaneous MEG-EEG recordings in unraveling the predictive, syntactic, and semantic mechanisms that underlie language comprehension.