PhysMamba: Efficient Remote Physiological Measurement with SlowFast Temporal Difference Mamba

TL;DR

PhysMamba introduces a novel Mamba-based framework with a Temporal Difference block and SlowFast architecture to efficiently capture long-range spatio-temporal dependencies in facial videos for remote physiological measurement, outperforming existing methods.

Contribution

The paper presents PhysMamba, the first to combine Mamba models with a dual-stream SlowFast architecture for efficient long-range dependency modeling in rPPG.

Findings

Outperforms existing CNN and Transformer-based methods on benchmark datasets.

Demonstrates high efficiency and accuracy in remote physiological measurement.

Effectively captures long-range spatio-temporal dependencies in facial videos.

Abstract

Facial-video based Remote photoplethysmography (rPPG) aims at measuring physiological signals and monitoring heart activity without any contact, showing significant potential in various applications. Previous deep learning based rPPG measurement are primarily based on CNNs and Transformers. However, the limited receptive fields of CNNs restrict their ability to capture long-range spatio-temporal dependencies, while Transformers also struggle with modeling long video sequences with high complexity. Recently, the state space models (SSMs) represented by Mamba are known for their impressive performance on capturing long-range dependencies from long sequences. In this paper, we propose the PhysMamba, a Mamba-based framework, to efficiently represent long-range physiological dependencies from facial videos. Specifically, we introduce the Temporal Difference Mamba block to first enhance local dynamic differences and further model the long-range spatio-temporal context. Moreover, a dual-stream SlowFast architecture is utilized to fuse the multi-scale temporal features. Extensive experiments are conducted on three benchmark datasets to demonstrate the superiority and efficiency of PhysMamba. The codes are available at https://github.com/Chaoqi31/PhysMamba