Learning the relative composition of EEG signals using pairwise relative shift pretraining

TL;DR

This paper introduces PARS, a novel self-supervised pretraining method for EEG signals that predicts relative temporal shifts, enabling models to learn long-range dependencies and improve performance on EEG decoding tasks.

Contribution

The paper presents PARS, a new pretraining task for EEG that emphasizes relative temporal shift prediction, enhancing long-range dependency learning over existing methods.

Findings

PARS outperforms existing pretraining strategies in EEG decoding tasks.

PARS improves label efficiency and transfer learning performance.

PARS captures long-range dependencies in neural signals.

Abstract

Self-supervised learning (SSL) offers a promising approach for learning electroencephalography (EEG) representations from unlabeled data, reducing the need for expensive annotations for clinical applications like sleep staging and seizure detection. While current EEG SSL methods predominantly use masked reconstruction strategies like masked autoencoders (MAE) that capture local temporal patterns, position prediction pretraining remains underexplored despite its potential to learn long-range dependencies in neural signals. We introduce PAirwise Relative Shift or PARS pretraining, a novel pretext task that predicts relative temporal shifts between randomly sampled EEG window pairs. Unlike reconstruction-based methods that focus on local pattern recovery, PARS encourages encoders to capture relative temporal composition and long-range dependencies inherent in neural signals. Through comprehensive evaluation on various EEG decoding tasks, we demonstrate that PARS-pretrained transformers consistently outperform existing pretraining strategies in label-efficient and transfer learning settings, establishing a new paradigm for self-supervised EEG representation learning.