UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding

TL;DR

UniMind leverages large language models and a novel neuro-language connector to achieve unified, multi-task brain decoding from EEG signals, significantly improving performance across diverse datasets.

Contribution

The paper introduces UniMind, a novel EEG foundation model that uses a neuro-language connector and task-aware query selection to enhance multi-task brain decoding.

Findings

Outperforms state-of-the-art models with 12% average gain

Successfully decodes neural patterns across ten datasets

Provides neuroscientific insights into neural functional correlations

Abstract

Decoding human brain activity from electroencephalography (EEG) signals is a central challenge at the intersection of neuroscience and artificial intelligence, enabling diverse applications in mental state assessment, clinical monitoring, and human-machine interaction. Recent efforts have extensively explored EEG-based brain foundation models for generalized brain decoding, employing large-scale training on multiple datasets. However, most of these attempts struggle with generalizability and fail to achieve satisfactory performance without task-specific tuning due to pronounced inherent heterogeneity among decoding tasks. To address these challenges, we present UniMind, a general-purpose EEG foundation model for unified multi-task brain decoding by uniquely unleashing the power of large language models to comprehend complex neural patterns. UniMind offers several advantages. First, we design a Neuro-Language Connector to bridge the modality gap between neural signals and large language models, distilling and transforming the spatiotemporal neural patterns of EEG data into representations understandable by language models. Second, a Task-aware Query Selection module is proposed to inject task-awareness into the cross-modal alignment by dynamically generating task-adaptive query tokens, enabling learning of task-relevant neural patterns across diverse tasks. Extensive experiments across ten datasets demonstrate that UniMind substantially outperforms state-of-the-art multi-task decoding models, with an average gain of 12 percent, while also offering valuable neuroscientific insights into neural functional correlations across tasks. The code is available at https://github.com/kaleidoyao/UniMind.