Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement

TL;DR

This paper introduces a new 3D facial surface-based representation and a graph neural network for improved remote physiological measurement from facial videos, achieving state-of-the-art results.

Contribution

The paper proposes the STGraph and MeshPhys, a novel surface-aligned spatiotemporal modeling framework for facial rPPG that enhances robustness and interpretability.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Surface-aligned features improve robustness and generalization.

Receptive field constraints act as a strong structural prior.

Abstract

Facial remote photoplethysmography (rPPG) methods estimate physiological signals by modeling subtle color changes on the 3D facial surface over time. However, existing methods fail to explicitly align their receptive fields with the 3D facial surface-the spatial support of the rPPG signal. To address this, we propose the Facial Spatiotemporal Graph (STGraph), a novel representation that encodes facial color and structure using 3D facial mesh sequences-enabling surface-aligned spatiotemporal processing. We introduce MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals. Across four benchmark datasets, MeshPhys achieves state-of-the-art or competitive performance in both intra- and cross-dataset settings. Ablation studies show that constraining the model's receptive field to the facial surface acts as a strong structural prior, and that surface-aligned, 3D-aware node features are critical for robustly encoding facial surface color. Together, the STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG, enabling robust, interpretable, and generalizable estimation. Code is available at https://samcantrill.github.io/facial-stgraph-rppg/ .