BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics

TL;DR

BrainStratify introduces a novel neural disentanglement framework that improves speech decoding accuracy from intracranial neural signals by identifying functional groups and disentangling neural dynamics, advancing brain-computer interface capabilities.

Contribution

This work presents a unified coarse-to-fine neural disentanglement framework combining spatial-temporal modeling and product quantization, a novel approach for decoding speech from intracranial signals.

Findings

Significantly outperforms previous speech decoding methods.

Effective in identifying functional neural groups.

Robust across multiple intracranial datasets.

Abstract

Decoding speech directly from neural activity is a central goal in brain-computer interface (BCI) research. In recent years, exciting advances have been made through the growing use of intracranial field potential recordings, such as stereo-ElectroEncephaloGraphy (sEEG) and ElectroCorticoGraphy (ECoG). These neural signals capture rich population-level activity but present key challenges: (i) task-relevant neural signals are sparsely distributed across sEEG electrodes, and (ii) they are often entangled with task-irrelevant neural signals in both sEEG and ECoG. To address these challenges, we introduce a unified Coarse-to-Fine neural disentanglement framework, BrainStratify, which includes (i) identifying functional groups through spatial-context-guided temporal-spatial modeling, and (ii) disentangling distinct neural dynamics within the target functional group using Decoupled Product Quantization (DPQ). We evaluate BrainStratify on two open-source sEEG datasets and one (epidural) ECoG dataset, spanning tasks like vocal production and speech perception. Extensive experiments show that BrainStratify, as a unified framework for decoding speech from intracranial neural signals, significantly outperforms previous decoding methods. Overall, by combining data-driven stratification with neuroscience-inspired modularity, BrainStratify offers a robust and interpretable solution for speech decoding from intracranial recordings.