MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing

TL;DR

MobilePhys is a novel mobile system that uses dual cameras on smartphones to generate self-supervised labels for personalized contactless physiological sensing, improving robustness and accuracy in real-world conditions.

Contribution

This paper introduces MobilePhys, the first mobile personalized contactless PPG system leveraging dual cameras and self-supervised learning to enhance model generalization without gold standard data.

Findings

MobilePhys outperforms state-of-the-art supervised and few-shot models.

The system maintains robustness across different devices, lighting, and skin types.

User studies confirm effectiveness in real-world scenarios.

Abstract

Camera-based contactless photoplethysmography refers to a set of popular techniques for contactless physiological measurement. The current state-of-the-art neural models are typically trained in a supervised manner using videos accompanied by gold standard physiological measurements. However, they often generalize poorly out-of-domain examples (i.e., videos that are unlike those in the training set). Personalizing models can help improve model generalizability, but many personalization techniques still require some gold standard data. To help alleviate this dependency, in this paper, we present a novel mobile sensing system called MobilePhys, the first mobile personalized remote physiological sensing system, that leverages both front and rear cameras on a smartphone to generate high-quality self-supervised labels for training personalized contactless camera-based PPG models. To evaluate the robustness of MobilePhys, we conducted a user study with 39 participants who completed a set of tasks under different mobile devices, lighting conditions/intensities, motion tasks, and skin types. Our results show that MobilePhys significantly outperforms the state-of-the-art on-device supervised training and few-shot adaptation methods. Through extensive user studies, we further examine how does MobilePhys perform in complex real-world settings. We envision that calibrated or personalized camera-based contactless PPG models generated from our proposed dual-camera mobile sensing system will open the door for numerous future applications such as smart mirrors, fitness and mobile health applications.