Spoken Speech Enhancement using EEG

TL;DR

This paper introduces a novel approach to speech enhancement using EEG signals and various neural network models, demonstrating significant improvements over traditional methods in noisy environments.

Contribution

First demonstration of spoken speech enhancement using EEG signals with multiple neural network models, showing improved quality over traditional algorithms.

Findings

EEG features can effectively clean speech in noisy settings.

GAN, GRU, TCN, and mixed models outperform traditional log MMSE.

First use of EEG for speech enhancement in parallel recordings.

Abstract

In this paper we demonstrate spoken speech enhancement using electroencephalography (EEG) signals using a generative adversarial network (GAN) based model, gated recurrent unit (GRU) regression based model, temporal convolutional network (TCN) regression model and finally using a mixed TCN GRU regression model. We compare our EEG based speech enhancement results with traditional log minimum mean-square error (MMSE) speech enhancement algorithm and our proposed methods demonstrate significant improvement in speech enhancement quality compared to the traditional method. Our overall results demonstrate that EEG features can be used to clean speech recorded in presence of background noise. To the best of our knowledge this is the first time a spoken speech enhancement is demonstrated using EEG features recorded in parallel with spoken speech.