PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring

TL;DR

PhysDrive is a large-scale, multimodal dataset designed for contactless in-vehicle physiological sensing, capturing diverse driving conditions and providing benchmarks to advance driver monitoring research.

Contribution

This paper introduces PhysDrive, the first comprehensive multimodal dataset for in-vehicle physiological monitoring with diverse modalities and real-world driving scenarios.

Findings

Established baseline performance for signal processing methods

Evaluated deep learning approaches across multiple modalities

Provided open-source tools for the research community

Abstract

Robust and unobtrusive in-vehicle physiological monitoring is crucial for ensuring driving safety and user experience. While remote physiological measurement (RPM) offers a promising non-invasive solution, its translation to real-world driving scenarios is critically constrained by the scarcity of comprehensive datasets. Existing resources are often limited in scale, modality diversity, the breadth of biometric annotations, and the range of captured conditions, thereby omitting inherent real-world challenges in driving. Here, we present PhysDrive, the first large-scale multimodal dataset for contactless in-vehicle physiological sensing with dedicated consideration on various modality settings and driving factors. PhysDrive collects data from 48 drivers, including synchronized RGB, near-infrared camera, and raw mmWave radar data, accompanied with six synchronized ground truths (ECG, BVP, Respiration, HR, RR, and SpO2). It covers a wide spectrum of naturalistic driving conditions, including driver motions, dynamic natural light, vehicle types, and road conditions. We extensively evaluate both signal-processing and deep-learning methods on PhysDrive, establishing a comprehensive benchmark across all modalities, and release full open-source code with compatibility for mainstream public toolboxes. We envision PhysDrive will serve as a foundational resource and accelerate research on multimodal driver monitoring and smart-cockpit systems.