A Hybrid End-to-End Spatio-Temporal Attention Neural Network with Graph-Smooth Signals for EEG Emotion Recognition

TL;DR

This paper introduces a hybrid deep neural network with spatio-temporal attention and graph smoothing for EEG-based emotion recognition, achieving state-of-the-art results and demonstrating transfer learning capabilities across datasets.

Contribution

The novel hybrid neural network architecture combines spatio-temporal encoding, recurrent attention, and graph signal processing for improved EEG emotion recognition.

Findings

Outperforms state-of-the-art on DEAP dataset

Effective transfer learning across different EEG datasets

Improved emotion classification accuracy with graph smoothing

Abstract

Recently, physiological data such as electroencephalography (EEG) signals have attracted significant attention in affective computing. In this context, the main goal is to design an automated model that can assess emotional states. Lately, deep neural networks have shown promising performance in emotion recognition tasks. However, designing a deep architecture that can extract practical information from raw data is still a challenge. Here, we introduce a deep neural network that acquires interpretable physiological representations by a hybrid structure of spatio-temporal encoding and recurrent attention network blocks. Furthermore, a preprocessing step is applied to the raw data using graph signal processing tools to perform graph smoothing in the spatial domain. We demonstrate that our proposed architecture exceeds state-of-the-art results for emotion classification on the publicly available DEAP dataset. To explore the generality of the learned model, we also evaluate the performance of our architecture towards transfer learning (TL) by transferring the model parameters from a specific source to other target domains. Using DEAP as the source dataset, we demonstrate the effectiveness of our model in performing cross-modality TL and improving emotion classification accuracy on DREAMER and the Emotional English Word (EEWD) datasets, which involve EEG-based emotion classification tasks with different stimuli.