Cross-Subject Emotion Recognition with Sparsely-Labeled Peripheral Physiological Data Using SHAP-Explained Tree Ensembles

TL;DR

This paper presents a novel approach for emotion recognition using sparsely-labeled physiological data, combining signal spectrum analysis, LightGBM, and SHAP for improved accuracy and interpretability, validated on the DEAP dataset.

Contribution

It introduces a spectrum analysis-based feature extraction method and applies SHAP explanations to LightGBM models for emotion recognition from physiological signals.

Findings

LightGBM outperformed XGBoost with higher F1-scores.

SHAP identified key features influencing emotion prediction.

The model provided insights into physiological-emotional relationships.

Abstract

There are still many challenges of emotion recognition using physiological data despite the substantial progress made recently. In this paper, we attempted to address two major challenges. First, in order to deal with the sparsely-labeled physiological data, we first decomposed the raw physiological data using signal spectrum analysis, based on which we extracted both complexity and energy features. Such a procedure helped reduce noise and improve feature extraction effectiveness. Second, in order to improve the explainability of the machine learning models in emotion recognition with physiological data, we proposed Light Gradient Boosting Machine (LightGBM) and SHapley Additive exPlanations (SHAP) for emotion prediction and model explanation, respectively. The LightGBM model outperformed the eXtreme Gradient Boosting (XGBoost) model on the public Database for Emotion Analysis using Physiological signals (DEAP) with f1-scores of 0.814, 0.823, and 0.860 for binary classification of valence, arousal, and liking, respectively, with cross-subject validation using eight peripheral physiological signals. Furthermore, the SHAP model was able to identify the most important features in emotion recognition, and revealed the relationships between the predictor variables and the response variables in terms of their main effects and interaction effects. Therefore, the results of the proposed model not only had good performance using peripheral physiological data, but also gave more insights into the underlying mechanisms in recognizing emotions.