SACM: SEEG-Audio Contrastive Matching for Chinese Speech Decoding

TL;DR

This study introduces SACM, a contrastive learning framework for Mandarin speech decoding using SEEG data, achieving high accuracy and revealing key brain regions, advancing speech neuroprosthesis development.

Contribution

The paper presents a novel SEEG-Audio Contrastive Matching framework for Chinese speech decoding, demonstrating effective decoding with minimal electrodes and providing insights for BCI improvements.

Findings

Decoding accuracies significantly exceed chance levels.

Single sensorimotor cortex electrode performs comparably to full array.

Provides insights for developing more accurate speech BCIs.

Abstract

Speech disorders such as dysarthria and anarthria can severely impair the patient's ability to communicate verbally. Speech decoding brain-computer interfaces (BCIs) offer a potential alternative by directly translating speech intentions into spoken words, serving as speech neuroprostheses. This paper reports an experimental protocol for Mandarin Chinese speech decoding BCIs, along with the corresponding decoding algorithms. Stereo-electroencephalography (SEEG) and synchronized audio data were collected from eight drug-resistant epilepsy patients as they conducted a word-level reading task. The proposed SEEG and Audio Contrastive Matching (SACM), a contrastive learning-based framework, achieved decoding accuracies significantly exceeding chance levels in both speech detection and speech decoding tasks. Electrode-wise analysis revealed that a single sensorimotor cortex electrode achieved performance comparable to that of the full electrode array. These findings provide valuable insights for developing more accurate online speech decoding BCIs.