Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks

TL;DR

This paper presents advanced deep neural network decoders for EEG responses to continuous speech, capable of accurately matching speech segments and generalizing across datasets, with potential applications in hearing disorder diagnosis and cognitive hearing aids.

Contribution

The study extends previous EEG decoders by analyzing speaker characteristics, providing comprehensive statistical performance evaluation, and demonstrating decoder generalization across different datasets.

Findings

Decoders achieve high classification accuracy in match-mismatch tasks.

Decoders can function as auditory attention decoders without retraining.

Robust performance across different speech-listening conditions.

Abstract

The electroencephalogram (EEG) offers a non-invasive means by which a listener's auditory system may be monitored during continuous speech perception. Reliable auditory-EEG decoders could facilitate the objective diagnosis of hearing disorders, or find applications in cognitively-steered hearing aids. Previously, we developed decoders for the ICASSP Auditory EEG Signal Processing Grand Challenge (SPGC). These decoders aimed to solve the match-mismatch task: given a short temporal segment of EEG recordings, and two candidate speech segments, the task is to identify which of the two speech segments is temporally aligned, or matched, with the EEG segment. The decoders made use of cortical responses to the speech envelope, as well as speech-related frequency-following responses, to relate the EEG recordings to the speech stimuli. Here we comprehensively document the methods by which the decoders were developed. We extend our previous analysis by exploring the association between speaker characteristics (pitch and sex) and classification accuracy, and provide a full statistical analysis of the final performance of the decoders as evaluated on a heldout portion of the dataset. Finally, the generalisation capabilities of the decoders are characterised, by evaluating them using an entirely different dataset which contains EEG recorded under a variety of speech-listening conditions. The results show that the match-mismatch decoders achieve accurate and robust classification accuracies, and they can even serve as auditory attention decoders without additional training.