Seismocardiography for Emotion Recognition: A Study on EmoWear with Insights from DEAP

TL;DR

This study explores the use of wearable accelerometers capturing seismocardiography and respiration signals for emotion recognition, demonstrating their viability as standalone modalities comparable to traditional cardiac signals, and enabling real-world affective computing.

Contribution

Introduces SCG and ADR as novel, standalone wearable modalities for emotion recognition, validated on the EmoWear dataset, and compares their performance with established cardiac signals.

Findings

SCG is a viable modality for emotion recognition, comparable to ECG and BVP.

Combining ADR with SCG enables emotion recognition with a single chest-worn accelerometer.

The proposed framework facilitates real-world affective computing with minimal hardware.

Abstract

Emotions have a profound impact on our daily lives, influencing our thoughts, behaviors, and interactions, but also our physiological reactions. Recent advances in wearable technology have facilitated studying emotions through cardio-respiratory signals. Accelerometers offer a non-invasive, convenient, and cost-effective method for capturing heart- and pulmonary-induced vibrations on the chest wall, specifically Seismocardiography (SCG) and Accelerometry-Derived Respiration (ADR). Their affordability, wide availability, and ability to provide rich contextual data make accelerometers ideal for everyday use. While accelerometers have been used as part of broader modality fusions for Emotion Recognition (ER), their stand-alone potential via SCG and ADR remains unexplored. Bridging this gap could significantly help the embedding of ER into real-world applications, minimizing the hardware, and increasing contextual integration potentials. To address this gap, we introduce SCG and ADR as novel modalities for ER and evaluate their performance using the EmoWear dataset. First, we replicate the single-trial emotion classification pipeline from the DEAP dataset study, achieving similar results. Then we use our validated pipeline to train models that predict affective valence-arousal states using SCG and compare them against established cardiac signals, Electrocardiography (ECG) and Blood Volume Pulse (BVP). Results show that SCG is a viable modality for ER, achieving similar performance to ECG and BVP. By combining ADR with SCG, we achieved a working ER framework that only requires a single chest-worn accelerometer. These findings pave the way for integrating ER into real-world, enabling seamless affective computing in everyday life.