Real-Time Monitoring of User Stress, Heart Rate and Heart Rate Variability on Mobile Devices

TL;DR

This paper presents Beam AI SDK, a real-time mobile solution that uses facial analysis to monitor user stress, heart rate, and variability with high accuracy, enabling apps to adapt content based on user emotional state.

Contribution

The introduction of a novel SDK that accurately measures stress and heart metrics via smartphone cameras in real-time, improving on existing methods.

Findings

Achieves over 97% accuracy in heart rate estimation on multiple datasets.

Demonstrates a strong correlation (0.801) between estimated and actual stress and HRV.

Provides a commercially viable SDK for real-time stress monitoring on mobile devices.

Abstract

Stress is considered to be the epidemic of the 21st-century. Yet, mobile apps cannot directly evaluate the impact of their content and services on user stress. We introduce the Beam AI SDK to address this issue. Using our SDK, apps can monitor user stress through the selfie camera in real-time. Our technology extracts the user's pulse wave by analyzing subtle color variations across the skin regions of the user's face. The user's pulse wave is then used to determine stress (according to the Baevsky Stress Index), heart rate, and heart rate variability. We evaluate our technology on the UBFC dataset, the MMSE-HR dataset, and Beam AI's internal data. Our technology achieves 99.2%, 97.8% and 98.5% accuracy for heart rate estimation on each benchmark respectively, a nearly twice lower error rate than competing methods. We further demonstrate an average Pearson correlation of 0.801 in determining stress and heart rate variability, thus producing commercially useful readings to derive content decisions in apps. Our SDK is available for use at www.beamhealth.ai.