Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction

TL;DR

Laya introduces a novel EEG foundation model based on latent prediction with JEPA, improving the encoding of meaningful neural signals and robustness over traditional reconstruction-based SSL methods.

Contribution

The paper presents Laya, the first EEG foundation model utilizing LeJEPA for latent prediction, enhancing representation quality and clinical task performance.

Findings

Laya embeddings effectively track seizure onset and clinical state changes.

Latent prediction yields more robust and semantically meaningful EEG representations.

Pretraining objective is the key factor driving performance improvements.

Abstract

Electroencephalography (EEG) is a widely used tool for studying brain function, with applications in clinical neuroscience, diagnosis, and brain-computer interfaces (BCIs). Recent EEG foundation models trained on large unlabeled corpora aim to learn transferable representations, but their effectiveness remains unclear; reported improvements over smaller task-specific models are often modest, sensitive to downstream adaptation and fine-tuning strategies, and limited under linear probing. We hypothesize that one contributing factor is the reliance on signal reconstruction as the primary self-supervised learning (SSL) objective, which biases representations toward high-variance artifacts rather than task-relevant neural structure. To address this limitation, we explore an SSL paradigm based on Joint Embedding Predictive Architectures (JEPA), which learn by predicting latent representations instead of reconstructing raw signals. We introduce Laya, the first EEG foundation model based on LeJEPA. We show that latent prediction yields representations that encode semantic structure in EEG: Laya embeddings track clinically meaningful state changes such as seizure onset, are resilient to noise, and achieve the strongest mean clinical accuracy under frozen linear probing, with particular gains on tasks where relevant neural patterns are subtle and easily obscured by artifacts. Controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver of these gains.