Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition

TL;DR

This paper introduces a deep learning approach that leverages EEG signals recorded during speech to significantly enhance automatic speech recognition for individuals with speech impairments like aphasia, apraxia, and dysarthria, aiming to develop real-time neural speech prosthetics.

Contribution

It presents a novel deep learning algorithm that utilizes synchronized EEG features to improve speech recognition accuracy for impaired speech, including initial results on continuous speech tasks.

Findings

Over 50% improvement in isolated speech recognition accuracy.

Preliminary positive results for continuous speech recognition.

Public release of a specialized EEG dataset for impaired speech.

Abstract

In this paper, we propose a deep learning-based algorithm to improve the performance of automatic speech recognition (ASR) systems for aphasia, apraxia, and dysarthria speech by utilizing electroencephalography (EEG) features recorded synchronously with aphasia, apraxia, and dysarthria speech. We demonstrate a significant decoding performance improvement by more than 50\% during test time for isolated speech recognition task and we also provide preliminary results indicating performance improvement for the more challenging continuous speech recognition task by utilizing EEG features. The results presented in this paper show the first step towards demonstrating the possibility of utilizing non-invasive neural signals to design a real-time robust speech prosthetic for stroke survivors recovering from aphasia, apraxia, and dysarthria. Our aphasia, apraxia, and dysarthria speech-EEG data set will be released to the public to help further advance this interesting and crucial research.