BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression

TL;DR

BrainRVQ is a novel EEG foundation model that employs dual-domain residual vector quantization and hierarchical autoregression to effectively capture neural dynamics, outperforming existing methods on multiple datasets.

Contribution

This work introduces BrainRVQ, a new EEG foundation model with a dual-domain quantization tokenizer and hierarchical autoregressive pre-training, addressing the hierarchical structure of neural signals.

Findings

Outperforms state-of-the-art EEG models on 8 datasets

Effectively captures hierarchical neural dynamics

Demonstrates robustness and generalization across diverse tasks

Abstract

Developing foundation models for electroencephalography (EEG) remains challenging due to the signal's low signal-to-noise ratio and complex spectro-temporal non-stationarity. Existing approaches often overlook the hierarchical latent structure inherent in neural dynamics, leading to suboptimal reconstruction of fine-grained information. In this work, we propose BrainRVQ, a general-purpose EEG foundation model pre-trained on a large-scale corpus of clinical EEG data. Unlike standard masked modeling, BrainRVQ features a Dual-Domain Residual Vector Quantization (DD-RVQ) tokenizer that disentangles temporal waveforms and spectral patterns into hierarchical discrete codes. We further introduce a hierarchical autoregressive pre-training objective that learns to reconstruct these codes in a coarse-to-fine manner, utilizing an importance-guided curriculum masking strategy to prioritize information-rich neural events over background noise. Extensive experiments across 8 diverse downstream datasets demonstrate that BrainRVQ consistently outperforms state-of-the-art baselines, validating its effectiveness in learning robust and generalizable neural representations. Our code and model weights are available:https://github.com/keqicmz/BrainRVQ