Enhancing Representation Learning of EEG Data with Masked Autoencoders

TL;DR

This paper introduces a masked autoencoder (MAE) for EEG data that improves representation learning and training efficiency, demonstrating its effectiveness on gaze estimation tasks and highlighting the potential of self-supervised learning in EEG applications.

Contribution

The study designs a novel MAE framework for EEG signals, showing significant improvements in learning efficiency and performance on downstream tasks.

Findings

MAE effectively learns EEG representations.

Pre-training with MAE reduces training time by two-thirds.

Pre-trained models outperform non-pre-trained models in EEG tasks.

Abstract

Self-supervised learning has been a powerful training paradigm to facilitate representation learning. In this study, we design a masked autoencoder (MAE) to guide deep learning models to learn electroencephalography (EEG) signal representation. Our MAE includes an encoder and a decoder. A certain proportion of input EEG signals are randomly masked and sent to our MAE. The goal is to recover these masked signals. After this self-supervised pre-training, the encoder is fine-tuned on downstream tasks. We evaluate our MAE on EEGEyeNet gaze estimation task. We find that the MAE is an effective brain signal learner. It also significantly improves learning efficiency. Compared to the model without MAE pre-training, the pre-trained one achieves equal performance with 1/3 the time of training and outperforms it in half the training time. Our study shows that self-supervised learning is a promising research direction for EEG-based applications as other fields (natural language processing, computer vision, robotics, etc.), and thus we expect foundation models to be successful in EEG domain.