BrainStack: Neuro-MoE with Functionally Guided Expert Routing for EEG-Based Language Decoding

TL;DR

BrainStack introduces a neuro-inspired mixture-of-experts framework for EEG-based language decoding, leveraging functional brain architecture and adaptive routing to improve accuracy and interpretability.

Contribution

This work presents BrainStack, a novel functionally guided Neuro-MoE model with expert routing and cross-regional distillation, advancing EEG-based language decoding methods.

Findings

Outperforms state-of-the-art models in accuracy and generalization

Introduces SilentSpeech-EEG, a large-scale EEG dataset for silent word decoding

Demonstrates the effectiveness of functional expert routing and hierarchical regularization

Abstract

Decoding linguistic information from electroencephalography (EEG) remains challenging due to the brain's distributed and nonlinear organization. We present BrainStack, a functionally guided neuro-mixture-of-experts (Neuro-MoE) framework that models the brain's modular functional architecture through anatomically partitioned expert networks. Each functional region is represented by a specialized expert that learns localized neural dynamics, while a transformer-based global expert captures cross-regional dependencies. A learnable routing gate adaptively aggregates these heterogeneous experts, enabling context-dependent expert coordination and selective fusion. To promote coherent representation across the hierarchy, we introduce cross-regional distillation, where the global expert provides top-down regularization to the regional experts. We further release SilentSpeech-EEG (SS-EEG), a large-scale benchmark comprising over 120 hours of EEG recordings from 12 subjects performing 24 silent words, the largest dataset of its kind. Experiments demonstrate that BrainStack consistently outperforms state-of-the-art models, achieving superior accuracy and generalization across subjects. Our results establish BrainStack as a functionally modular, neuro-inspired MoE paradigm that unifies neuroscientific priors with adaptive expert routing, paving the way for scalable and interpretable brain-language decoding.