Inter and Intra Signal Variance in Feature Extraction and Classification of Affective State

TL;DR

This study empirically evaluates how inter- and intra-signal variances in ECG and PPG data affect affective state classification, highlighting key features and challenges in physiological signal-based emotion recognition.

Contribution

It provides an empirical analysis of signal variances from wearables and their impact on affective state classification, identifying key features and state-specific challenges.

Findings

ECG and PPG feature variance affects classification performance

Beats-per-minute, inter-beat-interval, and breathing rate are key features

Certain affective states are harder to distinguish, requiring additional features

Abstract

Psychophysiology investigates the causal relationship of physiological changes resulting from psychological states. There are significant challenges with machine learning-based momentary assessments of physiology due to varying data collection methods, physiological differences, data availability and the requirement for expertly annotated data. Advances in wearable technology have significantly increased the scale, sensitivity and accuracy of devices for recording physiological signals, enabling large-scale unobtrusive physiological data gathering. This work contributes an empirical evaluation of signal variances acquired from wearables and their associated impact on the classification of affective states by (i) assessing differences occurring in features representative of affective states extracted from electrocardiograms and photoplethysmography, (ii) investigating the disparity in feature importance between signals to determine signal-specific features, and (iii) investigating the disparity in feature importance between affective states to determine affect-specific features. Results demonstrate that the degree of feature variance between ECG and PPG in a dataset is reflected in the classification performance of that dataset. Additionally, beats-per-minute, inter-beat-interval and breathing rate are identified as common best-performing features across both signals. Finally, feature variance per-affective state identifies hard-to-distinguish affective states requiring one-versus-rest or additional features to enable accurate classification.