Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing

TL;DR

This paper demonstrates that high-fidelity synthetic face videos with realistic blood flow and breathing patterns can improve the training of camera-based cardiopulmonary sensing algorithms, especially across diverse skin types.

Contribution

It introduces a synthetic data pipeline for generating physiologically-grounded face videos and systematically evaluates its impact on sensing accuracy and skin type diversity.

Findings

More synthetic avatars improve measurement accuracy.

Darker skin avatars enhance overall performance.

Synthetic data offers promising opportunities for physiological sensing.

Abstract

Synthetic data is a powerful tool in training data hungry deep learning algorithms. However, to date, camera-based physiological sensing has not taken full advantage of these techniques. In this work, we leverage a high-fidelity synthetics pipeline for generating videos of faces with faithful blood flow and breathing patterns. We present systematic experiments showing how physiologically-grounded synthetic data can be used in training camera-based multi-parameter cardiopulmonary sensing. We provide empirical evidence that heart and breathing rate measurement accuracy increases with the number of synthetic avatars in the training set. Furthermore, training with avatars with darker skin types leads to better overall performance than training with avatars with lighter skin types. Finally, we discuss the opportunities that synthetics present in the domain of camera-based physiological sensing and limitations that need to be overcome.