DELTA: Language Diffusion-based EEG-to-Text Architecture

TL;DR

DELTA introduces a novel EEG-to-text architecture combining a residual vector quantization tokenizer with a diffusion-based language model, significantly improving semantic alignment and text generation from EEG data.

Contribution

It is the first to integrate a residual vector quantization EEG tokenizer with a diffusion language model for EEG-to-text translation, addressing noise and variability issues.

Findings

Improves semantic alignment by up to 5.37 points over autoregressive models

Achieves BLEU-1 21.9 and ROUGE-1 F 17.2 on ZuCo dataset

Enables reliable text generation from small EEG-text datasets

Abstract

Electroencephalogram (EEG)-to-text remains challenging due to high-dimensional noise, subject variability, and error accumulation in autoregressive decoding. We introduce DELTA, which pairs a Residual Vector Quantization (RVQ) EEG tokenizer with a masked language diffusion model (LLaDA). RVQ discretizes continuous EEG into multi-layer tokens to reduce noise and individual differences, while LLaDA reconstructs sentences via non-sequential denoising. On ZuCo, DELTA improves semantic alignment by up to 5.37 points over autoregressive baselines, achieving BLEU-1 21.9 and ROUGE-1 F 17.2 under word-level conditions. These results enable reliable text generation from small EEG-text datasets and point toward scalable multimodal EEG-language models.