Toward Motion Robustness: A masked attention regularization framework in remote photoplethysmography

TL;DR

This paper introduces a masked attention regularization framework for remote photoplethysmography that enhances motion robustness and ROI localization accuracy, improving vital sign measurement in real-world facial video scenarios.

Contribution

It proposes a novel masked attention regularization method combined with an enhanced rPPG expert aggregation network to improve motion robustness and ROI localization in rPPG.

Findings

Outperforms state-of-the-art methods on three benchmark datasets.

Effectively mitigates motion artifacts and ROI localization errors.

Demonstrates superior generalization in real-world scenarios.

Abstract

There has been growing interest in facial video-based remote photoplethysmography (rPPG) measurement recently, with a focus on assessing various vital signs such as heart rate and heart rate variability. Despite previous efforts on static datasets, their approaches have been hindered by inaccurate region of interest (ROI) localization and motion issues, and have shown limited generalization in real-world scenarios. To address these challenges, we propose a novel masked attention regularization (MAR-rPPG) framework that mitigates the impact of ROI localization and complex motion artifacts. Specifically, our approach first integrates a masked attention regularization mechanism into the rPPG field to capture the visual semantic consistency of facial clips, while it also employs a masking technique to prevent the model from overfitting on inaccurate ROIs and subsequently degrading its performance. Furthermore, we propose an enhanced rPPG expert aggregation (EREA) network as the backbone to obtain rPPG signals and attention maps simultaneously. Our EREA network is capable of discriminating divergent attentions from different facial areas and retaining the consistency of spatiotemporal attention maps. For motion robustness, a simple open source detector MediaPipe for data preprocessing is sufficient for our framework due to its superior capability of rPPG signal extraction and attention regularization. Exhaustive experiments on three benchmark datasets (UBFC-rPPG, PURE, and MMPD) substantiate the superiority of our proposed method, outperforming recent state-of-the-art works by a considerable margin.