Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG

TL;DR

This paper introduces EEG-DisGCMAE, a novel pre-training framework that combines contrastive and masked autoencoder techniques to improve EEG classification by distilling knowledge from high-density to low-density data.

Contribution

It presents a unified graph self-supervised pre-training paradigm and a graph topology distillation loss, enabling effective transfer learning between high- and low-density EEG data.

Findings

Improves classification accuracy on clinical EEG datasets.

Effectively handles missing electrodes through contrastive distillation.

Outperforms existing methods in EEG transfer learning.

Abstract

Effectively utilizing extensive unlabeled high-density EEG data to improve performance in scenarios with limited labeled low-density EEG data presents a significant challenge. In this paper, we address this challenge by formulating it as a graph transfer learning and knowledge distillation problem. We propose a Unified Pre-trained Graph Contrastive Masked Autoencoder Distiller, named EEG-DisGCMAE, to bridge the gap between unlabeled and labeled as well as high- and low-density EEG data. Our approach introduces a novel unified graph self-supervised pre-training paradigm, which seamlessly integrates the graph contrastive pre-training with the graph masked autoencoder pre-training. Furthermore, we propose a graph topology distillation loss function, allowing a lightweight student model trained on low-density data to learn from a teacher model trained on high-density data during pre-training and fine-tuning. This method effectively handles missing electrodes through contrastive distillation. We validate the effectiveness of EEG-DisGCMAE across four classification tasks using two clinical EEG datasets with abundant data. The source code is available at https://github.com/weixinxu666/EEG_DisGCMAE.