State-of-the-art Speech Recognition using EEG and Towards Decoding of Speech Spectrum From EEG

TL;DR

This paper demonstrates the use of EEG signals for continuous noisy speech recognition with state-of-the-art models and explores decoding speech spectrum from EEG using LSTM and GAN, showing promising feasibility.

Contribution

It introduces methods for speech recognition from EEG and presents initial results on speech spectrum decoding, advancing EEG-based speech processing.

Findings

EEG signals enable continuous noisy speech recognition.

LSTM and GAN models can decode speech spectrum from EEG.

Preliminary results suggest feasibility of EEG-based speech synthesis.

Abstract

In this paper we first demonstrate continuous noisy speech recognition using electroencephalography (EEG) signals on English vocabulary using different types of state of the art end-to-end automatic speech recognition (ASR) models, we further provide results obtained using EEG data recorded under different experimental conditions. We finally demonstrate decoding of speech spectrum from EEG signals using a long short term memory (LSTM) based regression model and Generative Adversarial Network (GAN) based model. Our results demonstrate the feasibility of using EEG signals for continuous noisy speech recognition under different experimental conditions and we provide preliminary results for synthesis of speech from EEG features.