SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrieval

TL;DR

SENSE is a lightweight, privacy-preserving EEG-to-text framework that decodes brain signals into language without fine-tuning large models, using semantic retrieval and prompt-based generation.

Contribution

It introduces a novel two-stage EEG decoding approach that localizes processing and avoids exposing raw neural data, enhancing privacy and efficiency.

Findings

Achieves comparable or better text generation quality than fine-tuned baselines.

Reduces computational overhead by localizing decoding on-device.

Ensures neural data privacy by only sharing semantic cues.

Abstract

Decoding brain activity into natural language is a major challenge in AI with important applications in assistive communication, neurotechnology, and human-computer interaction. Most existing Brain-Computer Interface (BCI) approaches rely on memory-intensive fine-tuning of Large Language Models (LLMs) or encoder-decoder models on raw EEG signals, resulting in expensive training pipelines, limited accessibility, and potential exposure of sensitive neural data. We introduce SENSE (SEmantic Neural Sparse Extraction), a lightweight and privacy-preserving framework that translates non-invasive electroencephalography (EEG) into text without LLM fine-tuning. SENSE decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation. EEG signals are locally mapped to a discrete textual space to extract a non-sensitive Bag-of-Words (BoW), which conditions an off-the-shelf LLM to synthesize fluent text in a zero-shot manner. The EEG-to-keyword module contains only ~6M parameters and runs fully on-device, ensuring raw neural signals remain local while only abstract semantic cues interact with language models. Evaluated on a 128-channel EEG dataset across six subjects, SENSE matches or surpasses the generative quality of fully fine-tuned baselines such as Thought2Text while substantially reducing computational overhead. By localizing neural decoding and sharing only derived textual cues, SENSE provides a scalable and privacy-aware retrieval-augmented architecture for next-generation BCIs.