Heart rate and respiratory rate prediction from noisy real-world smartphone based on Deep Learning methods

TL;DR

This study evaluates the accuracy of smartphone-based vital sign estimation in real-world settings, revealing traditional methods perform poorly but deep learning significantly improves accuracy.

Contribution

It introduces a novel 3D deep CNN approach for estimating heart and respiratory rates from smartphone videos taken during daily life.

Findings

Traditional algorithms perform worse in real-world data.

Deep learning reduces HR estimation error by 68%.

Deep learning reduces RR estimation error by 75%.

Abstract

Using mobile phone video of the fingertip as a data source for estimating vital signs such as heart rate (HR) and respiratory rate (RR) during daily life has long been suggested. While existing literature indicates that these estimates are accurate to within several beats or breaths per minute, the data used to draw these conclusions are typically collected in laboratory environments under careful experimental control, and yet the results are assumed to generalize to daily life. In an effort to test it, a team of researchers collected a large dataset of mobile phone video recordings made during daily life and annotated with ground truth HR and RR labels from N=111 participants. They found that traditional algorithm performance on the fingerprint videos is worse than previously reported (7 times and 13 times worse for RR and HR, respectively). Fortunately, recent advancements in deep learning, especially in convolutional neural networks (CNNs), offer a promising solution to improve this performance. This study proposes a new method for estimating HR and RR using a novel 3D deep CNN, demonstrating a reduced error in estimated HR by 68% and RR by 75%. These promising results suggest that regressor-based deep learning approaches should be used in estimating HR and RR.