On Creating A Brain-To-Text Decoder

TL;DR

This paper explores decoding human speech from EEG signals using brain-computer interfaces, demonstrating improved accuracy with optimized electrode density, vocabulary size, and training data, advancing brain-to-text communication methods.

Contribution

It introduces a novel EEG-based brain-to-text decoding framework that leverages unlabelled data pre-training and analyzes key factors affecting BCI performance.

Findings

Achieved competitive word error rates on Librispeech benchmark.

Surpassed state-of-the-art voice recognition with limited labeled data.

Identified electrode density and vocabulary size as critical factors for BCI accuracy.

Abstract

Brain decoding has emerged as a rapidly advancing and extensively utilized technique within neuroscience. This paper centers on the application of raw electroencephalogram (EEG) signals for decoding human brain activity, offering a more expedited and efficient methodology for enhancing our understanding of the human brain. The investigation specifically scrutinizes the efficacy of brain-computer interfaces (BCI) in deciphering neural signals associated with speech production, with particular emphasis on the impact of vocabulary size, electrode density, and training data on the framework's performance. The study reveals the competitive word error rates (WERs) achievable on the Librispeech benchmark through pre-training on unlabelled data for speech processing. Furthermore, the study evaluates the efficacy of voice recognition under configurations with limited labeled data, surpassing previous state-of-the-art techniques while utilizing significantly fewer labels. Additionally, the research provides a comprehensive analysis of error patterns in voice recognition and the influence of model size and unlabelled training data. It underscores the significance of factors such as vocabulary size and electrode density in enhancing BCI performance, advocating for an increase in microelectrodes and refinement of language models.