BeatFormer: Efficient motion-robust remote heart rate estimation through unsupervised spectral zoomed attention filters

TL;DR

BeatFormer is a lightweight, unsupervised spectral attention model for remote heart rate estimation from facial videos, combining physiological priors and deep learning to improve robustness and efficiency in motion scenarios.

Contribution

It introduces a novel hybrid spectral attention model with spectral contrastive learning, enabling effective HR estimation without labeled data and enhancing motion robustness.

Findings

Demonstrates robustness across multiple datasets

Outperforms existing methods in motion scenarios

Operates efficiently with low computational cost

Abstract

Remote photoplethysmography (rPPG) captures cardiac signals from facial videos and is gaining attention for its diverse applications. While deep learning has advanced rPPG estimation, it relies on large, diverse datasets for effective generalization. In contrast, handcrafted methods utilize physiological priors for better generalization in unseen scenarios like motion while maintaining computational efficiency. However, their linear assumptions limit performance in complex conditions, where deep learning provides superior pulsatile information extraction. This highlights the need for hybrid approaches that combine the strengths of both methods. To address this, we present BeatFormer, a lightweight spectral attention model for rPPG estimation, which integrates zoomed orthonormal complex attention and frequency-domain energy measurement, enabling a highly efficient model. Additionally, we introduce Spectral Contrastive Learning (SCL), which allows BeatFormer to be trained without any PPG or HR labels. We validate BeatFormer on the PURE, UBFC-rPPG, and MMPD datasets, demonstrating its robustness and performance, particularly in cross-dataset evaluations under motion scenarios.