Magnetoencephalography (MEG) Based Non-Invasive Chinese Speech Decoding

TL;DR

This paper introduces a new MEG dataset and a multi-modality decoding algorithm for non-invasive Chinese speech BCI, demonstrating promising results and pioneering modality-assisted decoding in this domain.

Contribution

It provides the first Chinese speech MEG dataset and proposes a novel multi-modality assisted decoding algorithm for non-invasive speech BCIs.

Findings

Effective decoding of Chinese speech signals from MEG data

First demonstration of modality-assisted decoding for speech BCIs

Potential for improved communication aids for aphasia patients

Abstract

As an emerging paradigm of brain-computer interfaces (BCIs), speech BCI has the potential to directly reflect auditory perception and thoughts, offering a promising communication alternative for patients with aphasia. Chinese is one of the most widely spoken languages in the world, whereas there is very limited research on speech BCIs for Chinese language. This paper reports a text-magnetoencephalography (MEG) dataset for non-invasive Chinese speech BCIs. It also proposes a multi-modality assisted speech decoding (MASD) algorithm to capture both text and acoustic information embedded in brain signals during speech activities. Experiment results demonstrated the effectiveness of both our text-MEG dataset and our proposed MASD algorithm. To our knowledge, this is the first study on modality-assisted decoding for non-invasive speech BCIs.