# Predicting EEG Responses to Attended Speech via Deep Neural Networks for Speech

## TL;DR

This study demonstrates that deep neural network-extracted speech features significantly improve the prediction of EEG responses to attended speech in multi-talker environments, surpassing traditional hand-engineered features.

## Contribution

The paper introduces the use of hierarchical DNN-derived speech features to better predict neural responses and classify auditory attention from EEG data.

## Key findings

- DNN features outperform hand-engineered acoustic features in predicting EEG responses.
- Early DNN layers provide the highest neural response prediction accuracy.
- DNN features enhance auditory attention classification accuracy.

## Abstract

Attending to the speech stream of interest in multi-talker environments can be a challenging task, particularly for listeners with hearing impairment. Research suggests that neural responses assessed with electroencephalography (EEG) are modulated by listener`s auditory attention, revealing selective neural tracking (NT) of the attended speech. NT methods mostly rely on hand-engineered acoustic and linguistic speech features to predict the neural response. Only recently, deep neural network (DNN) models without specific linguistic information have been used to extract speech features for NT, demonstrating that speech features in hierarchical DNN layers can predict neural responses throughout the auditory pathway. In this study, we go one step further to investigate the suitability of similar DNN models for speech to predict neural responses to competing speech observed in EEG. We recorded EEG data using a 64-channel acquisition system from 17 listeners with normal hearing instructed to attend to one of two competing talkers. Our data revealed that EEG responses are significantly better predicted by DNN-extracted speech features than by hand-engineered acoustic features. Furthermore, analysis of hierarchical DNN layers showed that early layers yielded the highest predictions. Moreover, we found a significant increase in auditory attention classification accuracies with the use of DNN-extracted speech features over the use of hand-engineered acoustic features. These findings open a new avenue for development of new NT measures to evaluate and further advance hearing technology.