# Deep Learning the EEG Manifold for Phonological Categorization from Active Thoughts

## TL;DR

This paper introduces a novel deep learning-based BCI system that classifies phonological categories from EEG data, significantly improving accuracy and revealing discriminative speech imagery information.

## Contribution

It presents a hierarchical deep learning framework combining CNN, LSTM, and autoencoders for subject-independent phonological classification from EEG.

## Key findings

- Achieved 77.9% average accuracy in phonological classification
- Improved over previous methods by 22.5%
- Demonstrated discriminative information in speech imagery EEG

## Abstract

Speech-related Brain Computer Interfaces (BCI) aim primarily at finding an alternative vocal communication pathway for people with speaking disabilities. As a step towards full decoding of imagined speech from active thoughts, we present a BCI system for subject-independent classification of phonological categories exploiting a novel deep learning based hierarchical feature extraction scheme. To better capture the complex representation of high-dimensional electroencephalography (EEG) data, we compute the joint variability of EEG electrodes into a channel cross-covariance matrix. We then extract the spatio-temporal information encoded within the matrix using a mixed deep neural network strategy. Our model framework is composed of a convolutional neural network (CNN), a long-short term network (LSTM), and a deep autoencoder. We train the individual networks hierarchically, feeding their combined outputs in a final gradient boosting classification step. Our best models achieve an average accuracy of 77.9% across five different binary classification tasks, providing a significant 22.5% improvement over previous methods. As we also show visually, our work demonstrates that the speech imagery EEG possesses significant discriminative information about the intended articulatory movements responsible for natural speech synthesis.