Driver State Modeling through Latent Variable State Space Framework in the Wild

TL;DR

This paper introduces a latent-variable state-space framework to estimate drivers' stress and workload from multimodal sensor data, capturing environmental impacts and individual differences in real-world driving conditions.

Contribution

It presents a novel application of latent-variable state-space models for driver state estimation using multimodal data in naturalistic settings.

Findings

Latent stress and workload can be estimated from heart rate, gaze, and facial action units.

Environmental factors like traffic density influence driver stress and workload.

Drivers respond differently to environmental perturbations, with states showing temporal dependencies.

Abstract

Analyzing the impact of the environment on drivers' stress level and workload is of high importance for designing human-centered driver-vehicle interaction systems and to ultimately help build a safer driving experience. However, driver's state, including stress level and workload, are psychological constructs that cannot be measured on their own and should be estimated through sensor measurements such as psychophysiological measures. We propose using a latent-variable state-space modeling framework for driver state analysis. By using latent-variable state-space models, we model drivers' workload and stress levels as latent variables estimated through multimodal human sensing data, under the perturbations of the environment in a state-space format and in a holistic manner. Through using a case study of multimodal driving data collected from 11 participants, we first estimate the latent stress level and workload of drivers from their heart rate, gaze measures, and intensity of facial action units. We then show that external contextual elements such as the number of vehicles as a proxy for traffic density and secondary task demands may be associated with changes in driver's stress levels and workload. We also show that different drivers may be impacted differently by the aforementioned perturbations. We found out that drivers' latent states at previous timesteps are highly associated with their current states. Additionally, we discuss the utility of state-space models in analyzing the possible lag between the two constructs of stress level and workload, which might be indicative of information transmission between the different parts of the driver's psychophysiology in the wild.