Temporal Aware Mixed Attention-based Convolution and Transformer Network (MACTN) for EEG Emotion Recognition

TL;DR

This paper introduces MACTN, a hybrid neural network combining CNN and transformer architectures with attention mechanisms, designed for EEG-based emotion recognition, achieving superior accuracy on public datasets and winning a competition.

Contribution

The paper presents a novel hierarchical hybrid model that effectively captures local and global temporal features for EEG emotion recognition, inspired by neuroscience insights.

Findings

MACTN outperforms existing methods on THU-EP and DEAP datasets.

Integration of self-attention and channel attention improves classification accuracy.

An earlier version of MACTN won the 2022 Emotional BCI Competition.

Abstract

Emotion recognition plays a crucial role in human-computer interaction, and electroencephalography (EEG) is advantageous for reflecting human emotional states. In this study, we propose MACTN, a hierarchical hybrid model for jointly modeling local and global temporal information. The model is inspired by neuroscience research on the temporal dynamics of emotions. MACTN extracts local emotional features through a convolutional neural network (CNN) and integrates sparse global emotional features through a transformer. Moreover, we employ channel attention mechanisms to identify the most task-relevant channels. Through extensive experimentation on two publicly available datasets, namely THU-EP and DEAP, our proposed method, MACTN, consistently achieves superior classification accuracy and F1 scores compared to other existing methods in most experimental settings. Furthermore, ablation studies have shown that the integration of both self-attention mechanisms and channel attention mechanisms leads to improved classification performance. Finally, an earlier version of this method, which shares the same ideas, won the Emotional BCI Competition's final championship in the 2022 World Robot Contest.