PhysFlow: Skin tone transfer for remote heart rate estimation through conditional normalizing flows

TL;DR

PhysFlow is a novel data augmentation method using conditional normalizing flows to improve remote heart rate estimation across diverse skin tones, especially dark skin, by enhancing dataset diversity and reducing bias.

Contribution

It introduces PhysFlow, a new approach that augments skin tone diversity in datasets using conditional normalizing flows without requiring explicit skin-tone labels.

Findings

Reduced heart rate estimation error in dark skin tones.

Improved generalization of rPPG methods across skin tones.

Versatile application across different rPPG algorithms.

Abstract

In recent years, deep learning methods have shown impressive results for camera-based remote physiological signal estimation, clearly surpassing traditional methods. However, the performance and generalization ability of Deep Neural Networks heavily depends on rich training data truly representing different factors of variation encountered in real applications. Unfortunately, many current remote photoplethysmography (rPPG) datasets lack diversity, particularly in darker skin tones, leading to biased performance of existing rPPG approaches. To mitigate this bias, we introduce PhysFlow, a novel method for augmenting skin diversity in remote heart rate estimation using conditional normalizing flows. PhysFlow adopts end-to-end training optimization, enabling simultaneous training of supervised rPPG approaches on both original and generated data. Additionally, we condition our model using CIELAB color space skin features directly extracted from the facial videos without the need for skin-tone labels. We validate PhysFlow on publicly available datasets, UCLA-rPPG and MMPD, demonstrating reduced heart rate error, particularly in dark skin tones. Furthermore, we demonstrate its versatility and adaptability across different data-driven rPPG methods.