FacePhys: State of the Heart Learning

TL;DR

FacePhys introduces a memory-efficient, real-time rPPG algorithm that significantly improves accuracy and operational efficiency for remote vital sign measurement using cameras, enabling practical health monitoring applications.

Contribution

The paper presents FacePhys, a novel rPPG method based on temporal-spatial state space duality that achieves high accuracy with minimal computational resources and supports real-time deployment.

Findings

49% reduction in error over previous methods

Memory footprint of 3.6 MB enabling low-resource deployment

Per-frame latency of 9.46 ms supporting real-time inference

Abstract

Vital sign measurement using cameras presents opportunities for comfortable, ubiquitous health monitoring. Remote photoplethysmography (rPPG), a foundational technology, enables cardiac measurement through minute changes in light reflected from the skin. However, practical deployment is limited by the computational constraints of performing analysis on front-end devices and the accuracy degradation of transmitting data through compressive channels that reduce signal quality. We propose a memory efficient rPPG algorithm - \emph{FacePhys} - built on temporal-spatial state space duality, which resolves the trilemma of model scalability, cross-dataset generalization, and real-time operation. Leveraging a transferable heart state, FacePhys captures subtle periodic variations across video frames while maintaining a minimal computational overhead, enabling training on extended video sequences and supporting low-latency inference. FacePhys establishes a new state-of-the-art, with a substantial 49\% reduction in error. Our solution enables real-time inference with a memory footprint of 3.6 MB and per-frame latency of 9.46 ms -- surpassing existing methods by 83\% to 99\%. These results translate into reliable real-time performance in practical deployments, and a live demo is available at https://www.facephys.com/.