rPPG-VQA: A Video Quality Assessment Framework for Unsupervised rPPG Training

TL;DR

This paper introduces rPPG-VQA, a framework for assessing video quality specifically for unsupervised rPPG training, improving model performance by filtering suitable videos using signal and scene analyses.

Contribution

It presents a novel dual-branch assessment architecture combining signal-to-noise ratio estimation and multimodal large language model analysis for video suitability evaluation.

Findings

Filtering videos with rPPG-VQA improves unsupervised rPPG model accuracy.

The framework effectively identifies low-quality videos that degrade model training.

Using the proposed assessment, models trained on curated data outperform those trained on unfiltered data.

Abstract

Unsupervised remote photoplethysmography (rPPG) promises to leverage unlabeled video data, but its potential is hindered by a critical challenge: training on low-quality "in-the-wild" videos severely degrades model performance. An essential step missing here is to assess the suitability of the videos for rPPG model learning before using them for the task. Existing video quality assessment (VQA) methods are mainly designed for human perception and not directly applicable to the above purpose. In this work, we propose rPPG-VQA, a novel framework for assessing video suitability for rPPG. We integrate signal-level and scene-level analyses and design a dual-branch assessment architecture. The signal-level branch evaluates the physiological signal quality of the videos via robust signal-to-noise ratio (SNR) estimation with a multi-method consensus mechanism, and the scene-level branch uses a multimodal large language model (MLLM) to identify interferences like motion and unstable lighting. Furthermore, we propose a two-stage adaptive sampling (TAS) strategy that utilizes the quality score to curate optimal training datasets. Experiments show that by training on large-scale, "in-the-wild" videos filtered by our framework, we can develop unsupervised rPPG models that achieve a substantial improvement in accuracy on standard benchmarks. Our code is available at https://github.com/Tianyang-Dai/rPPG-VQA.