EEGDM: EEG Representation Learning via Generative Diffusion Model

TL;DR

This paper introduces EEGDM, a novel EEG representation learning framework using a structured diffusion model and transformer, which outperforms existing methods in epileptiform and seizure detection tasks.

Contribution

The paper proposes EEGDM, combining a structured state-space diffusion model and a latent fusion transformer for improved EEG representation learning.

Findings

EEGDM outperforms state-of-the-art EEG foundation models.

The structured diffusion pretraining captures EEG temporal dynamics effectively.

EEGDM achieves superior accuracy on epileptiform and seizure detection datasets.

Abstract

While electroencephalogram (EEG) has been a crucial tool for monitoring the brain and diagnosing neurological disorders (e.g., epilepsy), learning meaningful representations from raw EEG signals remains challenging due to limited annotations and high signal variability. Recently, EEG foundation models (FMs) have shown promising potential by adopting transformer architectures and self-supervised pre-training methods from large language models (e.g., masked prediction) to learn representations from diverse EEG data, followed by fine-tuning on specific EEG tasks. Nonetheless, these large models often incurred high computational costs during both training and inference, with only marginal performance improvements as the model size increases. In this work, we proposed an EEG representation learning framework building upon Generative Diffusion Model (EEGDM). Specifically, we developed a structured state-space model for diffusion pretraining (SSMDP) to better capture the temporal dynamics of EEG signals and trained it using Denoising Diffusion Probabilistic Model (DDPM) framework. Subsequently, the resulting latent EEG representations were then used for downstream classification tasks via our proposed latent fusion transformer (LFT). To evaluate our method, we used multi-event datasets covering both interictal epileptiform discharges (TUEV) and seizure (CHB-MIT) detection, and compared EEGDM with current state-of-the-art approaches, including EEG FMs. Empirical results showed that our method outperformed the existing methods. These findings suggested that EEGDM offered a promising alternative to current FMs. Our source code and checkpoint are available at: https://github.com/jhpuah/EEGDM.