RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement

TL;DR

RhythmMamba is a novel state space model-based method for remote photoplethysmography that efficiently captures long-range dependencies in facial videos, achieving high accuracy and throughput for physiological signal measurement.

Contribution

It introduces a state space model approach with multi-temporal and frequency domain constraints for improved rPPG signal extraction from videos.

Findings

Achieves state-of-the-art performance in rPPG measurement.

Provides 319% higher throughput and 23% lower peak GPU memory usage.

Effectively models long-range dependencies in facial video analysis.

Abstract

Remote photoplethysmography (rPPG) is a method for non-contact measurement of physiological signals from facial videos, holding great potential in various applications such as healthcare, affective computing, and anti-spoofing. Existing deep learning methods struggle to address two core issues of rPPG simultaneously: understanding the periodic pattern of rPPG among long contexts and addressing large spatiotemporal redundancy in video segments. These represent a trade-off between computational complexity and the ability to capture long-range dependencies. In this paper, we introduce RhythmMamba, a state space model-based method that captures long-range dependencies while maintaining linear complexity. By viewing rPPG as a time series task through the proposed frame stem, the periodic variations in pulse waves are modeled as state transitions. Additionally, we design multi-temporal constraint and frequency domain feed-forward, both aligned with the characteristics of rPPG time series, to improve the learning capacity of Mamba for rPPG signals. Extensive experiments show that RhythmMamba achieves state-of-the-art performance with 319% throughput and 23% peak GPU memory. The codes are available at https://github.com/zizheng-guo/RhythmMamba.