SympCam: Remote Optical Measurement of Sympathetic Arousal

TL;DR

SympCam introduces a novel 3D convolutional neural network with temporal attention for remote sympathetic arousal prediction from facial videos, significantly outperforming existing methods and providing a new dataset for the community.

Contribution

The paper presents SympCam, a new deep learning architecture with a temporal attention module for remote sympathetic arousal estimation, and releases the first dedicated dataset for this task.

Findings

SympCam improves sympathetic arousal prediction accuracy by 48%.

Our method detects physical stress with 90% balanced accuracy.

Compared to rPPG networks, SympCam significantly outperforms in arousal prediction.

Abstract

Recent work has shown that a person's sympathetic arousal can be estimated from facial videos alone using basic signal processing. This opens up new possibilities in the field of telehealth and stress management, providing a non-invasive method to measure stress only using a regular RGB camera. In this paper, we present SympCam, a new 3D convolutional architecture tailored to the task of remote sympathetic arousal prediction. Our model incorporates a temporal attention module (TAM) to enhance the temporal coherence of our sequential data processing capabilities. The predictions from our method improve accuracy metrics of sympathetic arousal in prior work by 48% to a mean correlation of 0.77. We additionally compare our method with common remote photoplethysmography (rPPG) networks and show that they alone cannot accurately predict sympathetic arousal "out-of-the-box". Furthermore, we show that the sympathetic arousal predicted by our method allows detecting physical stress with a balanced accuracy of 90% - an improvement of 61% compared to the rPPG method commonly used in related work, demonstrating the limitations of using rPPG alone. Finally, we contribute a dataset designed explicitly for the task of remote sympathetic arousal prediction. Our dataset contains synchronized face and hand videos of 20 participants from two cameras synchronized with electrodermal activity (EDA) and photoplethysmography (PPG) measurements. We will make this dataset available to the community and use it to evaluate the methods in this paper. To the best of our knowledge, this is the first dataset available to other researchers designed for remote sympathetic arousal prediction.