Enhancing the EEG Speech Match Mismatch Tasks With Word Boundaries

TL;DR

This paper improves EEG-based speech comprehension analysis by incorporating word boundary information, significantly boosting classification accuracy from 65-75% to 93%.

Contribution

It introduces a novel neural network approach that integrates word boundary cues into EEG-speech matching tasks, enhancing model performance.

Findings

Achieved 93% classification accuracy on a public dataset.

Demonstrated the importance of word boundary information in EEG speech processing.

Significantly outperformed previous methods with 65-75% accuracy.

Abstract

Recent studies have shown that the underlying neural mechanisms of human speech comprehension can be analyzed using a match-mismatch classification of the speech stimulus and the neural response. However, such studies have been conducted for fixed-duration segments without accounting for the discrete processing of speech in the brain. In this work, we establish that word boundary information plays a significant role in sentence processing by relating EEG to its speech input. We process the speech and the EEG signals using a network of convolution layers. Then, a word boundary-based average pooling is performed on the representations, and the inter-word context is incorporated using a recurrent layer. The experiments show that the modeling accuracy can be significantly improved (match-mismatch classification accuracy) to 93% on a publicly available speech-EEG data set, while previous efforts achieved an accuracy of 65-75% for this task.