Steering through Time: Blending Longitudinal Data with Simulation to Rethink Human-Autonomous Vehicle Interaction

TL;DR

This paper presents a hybrid framework combining longitudinal mobile sensing and high-fidelity simulation to study driver readiness and human-vehicle interaction in semi-automated vehicles, emphasizing personalized monitoring.

Contribution

It introduces a novel multimodal data collection approach linking long-term physiological data with real-time simulation responses for improved driver state assessment.

Findings

Feasibility demonstrated for multimodal data collection in driver studies.

Individual variability observed in physiological and behavioral measures.

Certain measures like fixation duration differed by task type and showed stability.

Abstract

As semi-automated vehicles (SAVs) become more common, ensuring effective human-vehicle interaction during control handovers remains a critical safety challenge. Existing studies often rely on single-session simulator experiments or naturalistic driving datasets, which often lack temporal context on drivers' cognitive and physiological states before takeover events. This study introduces a hybrid framework combining longitudinal mobile sensing with high-fidelity driving simulation to examine driver readiness in semi-automated contexts. In a pilot study with 38 participants, we collected 7 days of wearable physiological data and daily surveys on stress, arousal, valence, and sleep quality, followed by an in-lab simulation with scripted takeover events under varying secondary task conditions. Multimodal sensing, including eye tracking, fNIRS, and physiological measures, captured real-time responses. Preliminary analysis shows the framework's feasibility and individual variability in baseline and in-task measures; for example, fixation duration and takeover control time differed by task type, and RMSSD showed high inter-individual stability. This proof-of-concept supports the development of personalized, context-aware driver monitoring by linking temporally layered data with real-time performance.