TRUCE-AV: A Multimodal dataset for Trust and Comfort Estimation in Autonomous Vehicles

TL;DR

This paper introduces TRUCE-AV, a comprehensive multimodal dataset capturing physiological, environmental, and subjective data to estimate trust and comfort in autonomous vehicles, facilitating improved human-centered AV systems.

Contribution

The paper presents a novel multimodal dataset for real-time trust and comfort estimation in autonomous driving, combining physiological signals, environmental data, and subjective assessments.

Findings

Tree-based models like Random Forest and XGBoost perform best for trust and comfort estimation.

SHAP analysis identifies key physiological and environmental features influencing trust and comfort.

The dataset enables development of adaptive AV systems that respond to user trust and comfort levels.

Abstract

Understanding and estimating driver trust and comfort are essential for the safety and widespread acceptance of autonomous vehicles. Existing works analyze user trust and comfort separately, with limited real-time assessment and insufficient multimodal data. This paper introduces a novel multimodal dataset called TRUCE-AV, focusing on trust and comfort estimation in autonomous vehicles. The dataset collects real-time trust votes and continuous comfort ratings of 31 participants during a simulator-based fully autonomous driving. Simultaneously, physiological signals, such as heart rate, gaze, and emotions, along with environmental data (e.g., vehicle speed, nearby vehicle positions, and velocity), are recorded throughout the drives. Standard pre- and post-drive questionnaires were also administered to assess participants' trust in automation and overall well-being, enabling the correlation of subjective assessments with real-time responses. To demonstrate the utility of our dataset, we evaluated various machine learning models for trust and comfort estimation using physiological data. Our analysis showed that tree-based models like Random Forest and XGBoost and non-linear models such as KNN and MLP regressor achieved the best performance for trust classification and comfort regression. Additionally, we identified key features that contribute to these estimations by using SHAP analysis on the top-performing models. Our dataset enables the development of adaptive AV systems capable of dynamically responding to user trust and comfort levels non-invasively, ultimately enhancing safety, user experience, and human-centered vehicle design.