Brain-CLIPLM: Decoding Compressed Semantic Representations in EEG for Language Reconstruction

TL;DR

This paper introduces Brain-CLIPLM, a novel framework for decoding semantic representations from EEG signals into language, emphasizing semantic compression over full sentence reconstruction, and demonstrates its effectiveness on a cognitive language corpus.

Contribution

The work proposes a two-stage EEG-to-text decoding method that aligns with neural information capacity, improving accuracy and robustness over direct decoding approaches.

Findings

Achieves 67.55% top-5 sentence retrieval accuracy

Significantly outperforms baseline in EEG-to-text decoding

EEG representations encode sentence-specific semantic content

Abstract

Decoding natural language from non-invasive electroencephalography (EEG) remains fundamentally limited by low signal-to-noise ratio and restricted information bandwidth. This raises a fundamental question regarding whether sentence-level linguistic structure can be reliably recovered from such signals. In this work, we suggest that this assumption may not hold under realistic information constraints, and instead propose a semantic compression hypothesis in which EEG signals encode a compressed set of semantic anchors rather than full linguistic structure. Under our new perspective, direct sentence reconstruction becomes an overparameterized objective relative to the intrinsic information capacity of EEG. To address this mismatch, we introduce Brain-CLIPLM, a two-stage framework that decomposes EEG-to-text decoding into semantic anchor extraction via contrastive learning and sentence reconstruction using a retrieval-grounded large language model (LLM) with Chain-of-Thought (CoT) reasoning, following a granularity matching principle that aligns decoding complexity with neural information capacity. Evaluated on the Zurich Cognitive Language Processing Corpus, Brain-CLIPLM achieves 67.55\% top-5 and 85.00\% top-25 sentence retrieval accuracy, significantly outperforming direct decoding baseline, while cross-subject evaluation confirms robust generalization. Control analyses, including permutation testing, further demonstrate that EEG-derived representations carry sentence-specific information beyond language model priors. These results suggest that EEG-to-text decoding is better framed as recovering compressed semantic content rather than reconstructing full sentences, providing a biologically grounded and data-efficient pathway for non-invasive brain-computer interfaces.