Decoding Predictive Inference in Visual Language Processing via Spatiotemporal Neural Coherence

TL;DR

This study introduces a machine learning approach that decodes neural responses to visual language stimuli in Deaf signers, revealing neural coherence patterns linked to predictive language processing.

Contribution

It presents a novel multimodal framework combining EEG and optical flow features to analyze neural dynamics during visual language comprehension.

Findings

Distributed left-hemispheric and frontal low-frequency coherence are key in language understanding.

Experience-dependent neural signatures correlate with age.

Identifies frequency-specific neural signatures differentiating meaningful from disrupted stimuli.

Abstract

Human language processing relies on the brain's capacity for predictive inference. We present a machine learning framework for decoding neural (EEG) responses to dynamic visual language stimuli in Deaf signers. Using coherence between neural signals and optical flow-derived motion features, we construct spatiotemporal representations of predictive neural dynamics. Through entropy-based feature selection, we identify frequency-specific neural signatures that differentiate interpretable linguistic input from linguistically disrupted (time-reversed) stimuli. Our results reveal distributed left-hemispheric and frontal low-frequency coherence as key features in language comprehension, with experience-dependent neural signatures correlating with age. This work demonstrates a novel multimodal approach for probing experience-driven generative models of perception in the brain.