Enhancing Generalization in PPG-Based Emotion Measurement with a CNN-TCN-LSTM Model

TL;DR

This paper presents a hybrid CNN-TCN-LSTM model that significantly improves the generalization of PPG-based emotion recognition across individuals, outperforming existing models in accuracy and robustness.

Contribution

The study introduces a novel hybrid deep learning architecture combining CNN, TCN, and LSTM to enhance cross-subject emotion recognition from PPG signals.

Findings

The hybrid model outperforms standalone CNN and CNN-LSTM models.

It achieves higher AUC and F1 scores in emotion classification.

The approach demonstrates improved robustness to individual variability.

Abstract

Human computer interaction has become integral to modern life, driven by advancements in machine learning technologies. Affective computing, in particular, has focused on systems that recognize, interpret, and respond to human emotions, often using wearable devices, which provide continuous data streams of physiological signals. Among various physiological signals, the photoplethysmogram (PPG) has gained prominence due to its ease of acquisition from widely available devices. However, the generalization of PPG-based emotion recognition models across individuals remains an unresolved challenge. This paper introduces a novel hybrid architecture that combines Convolutional Neural Networks (CNNs), Long Short-Term Memory networks (LSTMs), and Temporal Convolutional Networks (TCNs) to address this issue. The proposed model integrates the strengths of these architectures to improve robustness and generalization. Raw PPG signals are fed into the CNN for feature extraction. These features are processed separately by LSTM and TCN. The outputs from these components are concatenated to generate a final feature representation, which serves as the input for classifying valence and arousal, the primary dimensions of emotion. Experiments using the Photoplethysmogram Dataset for Emotional Analysis (PPGE) demonstrate that the proposed hybrid model achieves better model generalization than standalone CNN and LSTM architectures. Our results show that the proposed solution outperforms the state-of-the-art CNN architecture, as well as a CNN-LSTM model, in emotion recognition tasks with PPG signals. Using metrics such as Area Under the Curve (AUC) and F1 Score, we highlight the model's effectiveness in handling subject variability.