How Well Can We Decode Vowels from Auditory EEG -- A Rigorous Cross-Subject Benchmark with Honest Assessment

TL;DR

This study establishes a rigorous cross-subject benchmark for decoding vowels from auditory EEG, comparing classical and deep learning methods under strict evaluation to assess decoding accuracy and underlying neural signals.

Contribution

It introduces a reproducible benchmark with strict evaluation protocols for vowel decoding from EEG, comparing multiple pipelines and analyzing neural signal characteristics.

Findings

Best model (XGBoost) achieved 24.5% accuracy, above chance.

Differential entropy features with LightGBM reached 25.5% accuracy.

Classical methods perform comparably to deep models in low signal regimes.

Abstract

EEG based phoneme decoding is promising for brain computer interfaces, but many prior studies rely on within subject evaluation, small cohorts, or weak leakage control. We present a reproducible cross subject benchmark for five class vowel decoding (a, e, i, o, u) from auditory EEG using OpenNeuro ds006104 (16 subjects, 61 channels, 256 Hz). Under strict leave one subject out evaluation with training only normalization and explicit anti leakage checks, we compare 14 pipelines from classical machine learning, deep learning, and Riemannian methods. The best full feature model (XGBoost) reaches 24.5 percent accuracy (chance 20 percent), while differential entropy features with LightGBM reach 25.5 percent in feature specific analysis. After multiple comparison correction, strong pairwise model advantages are limited. Classical methods are competitive with deep models in this low signal regime. Additional analyses (ablation, pairwise vowels, within subject CV, ERP, temporal generalization, and electrode importance) indicate that vowel information is real but weak and mainly carried by early transient auditory responses. We release code and evaluation scripts for full reproducibility.