Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition

TL;DR

This paper introduces CLISA, a contrastive learning approach that aligns EEG representations across subjects to improve cross-subject emotion recognition, achieving state-of-the-art results and providing neural insights.

Contribution

The paper presents a novel contrastive learning method for inter-subject EEG alignment, enhancing emotion recognition across different individuals.

Findings

Achieved state-of-the-art cross-subject emotion recognition accuracy.

Generalized well to unseen subjects and stimuli.

Provided neural insights through learned spatiotemporal representations.

Abstract

EEG signals have been reported to be informative and reliable for emotion recognition in recent years. However, the inter-subject variability of emotion-related EEG signals still poses a great challenge for the practical applications of EEG-based emotion recognition. Inspired by recent neuroscience studies on inter-subject correlation, we proposed a Contrastive Learning method for Inter-Subject Alignment (CLISA) to tackle the cross-subject emotion recognition problem. Contrastive learning was employed to minimize the inter-subject differences by maximizing the similarity in EEG signal representations across subjects when they received the same emotional stimuli in contrast to different ones. Specifically, a convolutional neural network was applied to learn inter-subject aligned spatiotemporal representations from EEG time series in contrastive learning. The aligned representations were subsequently used to extract differential entropy features for emotion classification. CLISA achieved state-of-the-art cross-subject emotion recognition performance on our THU-EP dataset with 80 subjects and the publicly available SEED dataset with 15 subjects. It could generalize to unseen subjects or unseen emotional stimuli in testing. Furthermore, the spatiotemporal representations learned by CLISA could provide insights into the neural mechanisms of human emotion processing.