Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks

TL;DR

This paper introduces a novel deep spatio-temporal network approach for accurately reconstructing rPPG signals from facial videos, enabling detailed heart rate variability analysis and applications like atrial fibrillation detection.

Contribution

It is the first to use deep spatio-temporal networks for precise rPPG signal reconstruction from raw facial videos, improving HRV analysis capabilities.

Findings

Achieves superior HR and HRV measurement accuracy compared to state-of-the-art methods.

Demonstrates effective use of reconstructed rPPG signals for AF detection.

Shows promising results in emotion recognition from facial videos.

Abstract

Recent studies demonstrated that the average heart rate (HR) can be measured from facial videos based on non-contact remote photoplethysmography (rPPG). However for many medical applications (e.g., atrial fibrillation (AF) detection) knowing only the average HR is not sufficient, and measuring precise rPPG signals from face for heart rate variability (HRV) analysis is needed. Here we propose an rPPG measurement method, which is the first work to use deep spatio-temporal networks for reconstructing precise rPPG signals from raw facial videos. With the constraint of trend-consistency with ground truth pulse curves, our method is able to recover rPPG signals with accurate pulse peaks. Comprehensive experiments are conducted on two benchmark datasets, and results demonstrate that our method can achieve superior performance on both HR and HRV levels comparing to the state-of-the-art methods. We also achieve promising results of using reconstructed rPPG signals for AF detection and emotion recognition.