EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG

TL;DR

This study introduces EEG2TEXT-CN, an innovative framework that aligns Chinese EEG signals with text using contrastive learning and pretrained language models, demonstrating initial feasibility for brain-to-text translation in Chinese.

Contribution

It presents one of the first open-vocabulary EEG-to-text models for Chinese, combining a biologically grounded EEG encoder with a pretrained language model for cross-modal alignment.

Findings

Achieved a BLEU-1 score of 6.38% in zero-shot sentence prediction.

Demonstrated the feasibility of Chinese brain-to-text translation.

Laid groundwork for future multilingual brain-language interfaces.

Abstract

We propose EEG2TEXT-CN, which, to the best of our knowledge, represents one of the earliest open-vocabulary EEG-to-text generation frameworks tailored for Chinese. Built on a biologically grounded EEG encoder (NICE-EEG) and a compact pretrained language model (MiniLM), our architecture aligns multichannel brain signals with natural language representations via masked pretraining and contrastive learning. Using a subset of the ChineseEEG dataset, where each sentence contains approximately ten Chinese characters aligned with 128-channel EEG recorded at 256 Hz, we segment EEG into per-character embeddings and predict full sentences in a zero-shot setting. The decoder is trained with teacher forcing and padding masks to accommodate variable-length sequences. Evaluation on over 1,500 training-validation sentences and 300 held-out test samples shows promising lexical alignment, with a best BLEU-1 score of 6.38\%. While syntactic fluency remains a challenge, our findings demonstrate the feasibility of non-phonetic, cross-modal language decoding from EEG. This work opens a new direction in multilingual brain-to-text research and lays the foundation for future cognitive-language interfaces in Chinese.