Joint Contrastive Learning with Feature Alignment for Cross-Corpus EEG-based Emotion Recognition

TL;DR

This paper introduces JCFA, a novel framework that uses contrastive learning and feature alignment to improve cross-corpus EEG-based emotion recognition, achieving state-of-the-art results across datasets.

Contribution

The study proposes a joint contrastive learning framework with feature alignment specifically designed for cross-corpus EEG emotion recognition, addressing variability across datasets.

Findings

Achieves 4.09% higher accuracy than previous methods.

Effectively aligns time-frequency EEG features across datasets.

Demonstrates robustness in cross-corpus emotion recognition.

Abstract

The integration of human emotions into multimedia applications shows great potential for enriching user experiences and enhancing engagement across various digital platforms. Unlike traditional methods such as questionnaires, facial expressions, and voice analysis, brain signals offer a more direct and objective understanding of emotional states. However, in the field of electroencephalography (EEG)-based emotion recognition, previous studies have primarily concentrated on training and testing EEG models within a single dataset, overlooking the variability across different datasets. This oversight leads to significant performance degradation when applying EEG models to cross-corpus scenarios. In this study, we propose a novel Joint Contrastive learning framework with Feature Alignment (JCFA) to address cross-corpus EEG-based emotion recognition. The JCFA model operates in two main stages. In the pre-training stage, a joint domain contrastive learning strategy is introduced to characterize generalizable time-frequency representations of EEG signals, without the use of labeled data. It extracts robust time-based and frequency-based embeddings for each EEG sample, and then aligns them within a shared latent time-frequency space. In the fine-tuning stage, JCFA is refined in conjunction with downstream tasks, where the structural connections among brain electrodes are considered. The model capability could be further enhanced for the application in emotion detection and interpretation. Extensive experimental results on two well-recognized emotional datasets show that the proposed JCFA model achieves state-of-the-art (SOTA) performance, outperforming the second-best method by an average accuracy increase of 4.09% in cross-corpus EEG-based emotion recognition tasks.