TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition

TL;DR

TSception is a novel multi-scale CNN that effectively captures temporal dynamics and spatial asymmetry in EEG signals, leading to improved emotion recognition accuracy across multiple datasets.

Contribution

It introduces a multi-scale convolutional architecture with dynamic temporal and asymmetric spatial layers specifically designed for EEG-based emotion recognition.

Findings

Achieves higher accuracy and F1 scores than prior methods on DEAP and MAHNOB-HCI datasets.

Effectively learns dynamic temporal and spatial features relevant to emotional states.

Demonstrates the importance of modeling both temporal dynamics and spatial asymmetry in EEG analysis.

Abstract

The high temporal resolution and the asymmetric spatial activations are essential attributes of electroencephalogram (EEG) underlying emotional processes in the brain. To learn the temporal dynamics and spatial asymmetry of EEG towards accurate and generalized emotion recognition, we propose TSception, a multi-scale convolutional neural network that can classify emotions from EEG. TSception consists of dynamic temporal, asymmetric spatial, and high-level fusion layers, which learn discriminative representations in the time and channel dimensions simultaneously. The dynamic temporal layer consists of multi-scale 1D convolutional kernels whose lengths are related to the sampling rate of EEG, which learns the dynamic temporal and frequency representations of EEG. The asymmetric spatial layer takes advantage of the asymmetric EEG patterns for emotion, learning the discriminative global and hemisphere representations. The learned spatial representations will be fused by a high-level fusion layer. Using more generalized cross-validation settings, the proposed method is evaluated on two publicly available datasets DEAP and MAHNOB-HCI. The performance of the proposed network is compared with prior reported methods such as SVM, KNN, FBFgMDM, FBTSC, Unsupervised learning, DeepConvNet, ShallowConvNet, and EEGNet. TSception achieves higher classification accuracies and F1 scores than other methods in most of the experiments. The codes are available at https://github.com/yi-ding-cs/TSception