Robust and Generalizable Heart Rate Estimation via Deep Learning for Remote Photoplethysmography in Complex Scenarios

TL;DR

This paper introduces a deep learning-based remote photoplethysmography (rPPG) method that improves heart rate estimation accuracy, robustness, and generalization in complex scenarios through innovative network modules and a new loss function.

Contribution

The paper presents an end-to-end rPPG extraction network with differential frame fusion, TSM with self-attention, and a dynamic hybrid loss, enhancing performance over existing models.

Findings

Achieved MAE of 7.58 on MMPD dataset, outperforming state-of-the-art.

Demonstrated robustness and generalization across multiple datasets.

Effective in complex real-world scenarios.

Abstract

Non-contact remote photoplethysmography (rPPG) technology enables heart rate measurement from facial videos. However, existing network models still face challenges in accu racy, robustness, and generalization capability under complex scenarios. This paper proposes an end-to-end rPPG extraction network that employs 3D convolutional neural networks to reconstruct accurate rPPG signals from raw facial videos. We introduce a differential frame fusion module that integrates differential frames with original frames, enabling frame-level representations to capture blood volume pulse (BVP) variations. Additionally, we incorporate Temporal Shift Module (TSM) with self-attention mechanisms, which effectively enhance rPPG features with minimal computational overhead. Furthermore, we propose a novel dynamic hybrid loss function that provides stronger supervision for the network, effectively mitigating over fitting. Comprehensive experiments were conducted on not only the PURE and UBFC-rPPG datasets but also the challenging MMPD dataset under complex scenarios, involving both intra dataset and cross-dataset evaluations, which demonstrate the superior robustness and generalization capability of our network. Specifically, after training on PURE, our model achieved a mean absolute error (MAE) of 7.58 on the MMPD test set, outperforming the state-of-the-art models.