A High-Throughput Platform to Bench Test Smartphone-Based Heart Rate Measurements Derived From Video

TL;DR

This paper introduces a scalable, high-throughput bench-testing platform for evaluating smartphone-based heart rate measurement apps using synthetic videos, enabling standardized, rapid performance assessment across multiple devices.

Contribution

The paper presents a novel, automated testing system that allows parallel evaluation of multiple smartphones with synthetic PPG videos, addressing device variability and standardization challenges.

Findings

Achieved a mean absolute percentage error of 0.11% in HR measurement.

Correctly classified all tested devices as meeting ANSI/CTA standards.

Demonstrated high correlation (0.92) between input and measured PPG signals.

Abstract

Smartphone-based heart rate (HR) monitoring apps using finger-over-camera photoplethysmography (PPG) face significant challenges in performance evaluation and device compatibility due to device variability and fragmentation. Manual testing is impractical, and standardized methods are lacking. This paper presents a novel, high-throughput bench-testing platform to address this critical need. We designed a system comprising a test rig capable of holding 12 smartphones for parallel testing, a method for generating synthetic PPG test videos with controllable HR and signal quality, and a host machine for coordinating video playback and data logging. The system achieved a mean absolute percentage error (MAPE) of 0.11% +/- 0.001% between input and measured HR, and a correlation coefficient of 0.92 +/- 0.008 between input and measured PPG signals using a clinically-validated smartphone-based HR app. Bench-testing results of 20 different smartphone models correctly classified all the devices as meeting the ANSI/CTA accuracy standards for HR monitors (MAPE <10%) when compared to a prospective clinical study with 80 participants, demonstrating high positive predictive value. This platform offers a scalable solution for pre-deployment testing of smartphone HR apps to improve app performance, ensure device compatibility, and advance the field of mobile health.