A Virtual Reality Simulation Pipeline for Online Mental Workload Modeling
Robert L. Wilson, Daniel Browne, Jonathan Wagstaff, and Steve McGuire

TL;DR
This paper presents a novel VR simulation pipeline that integrates multimodal biosignal data for real-time mental workload modeling, enhancing human-robot interaction and adaptive system responses in high-stress scenarios.
Contribution
It introduces a VR system inspired by NASA's MATB-II that synchronously collects performance and biosignal data for online mental workload estimation.
Findings
Developed a VR pipeline capable of real-time biosignal collection and processing.
Facilitated integration of psychophysiological data into MW models via ROS.
Enabled adaptive responses in VR environments based on operator mental workload.
Abstract
Seamless human robot interaction (HRI) and cooperative human-robot (HR) teaming critically rely upon accurate and timely human mental workload (MW) models. Cognitive Load Theory (CLT) suggests representative physical environments produce representative mental processes; physical environment fidelity corresponds with improved modeling accuracy. Virtual Reality (VR) systems provide immersive environments capable of replicating complicated scenarios, particularly those associated with high-risk, high-stress scenarios. Passive biosignal modeling shows promise as a noninvasive method of MW modeling. However, VR systems rarely include multimodal psychophysiological feedback or capitalize on biosignal data for online MW modeling. Here, we develop a novel VR simulation pipeline, inspired by the NASA Multi-Attribute Task Battery II (MATB-II) task architecture, capable of synchronous collection…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman-Automation Interaction and Safety · Healthcare Technology and Patient Monitoring · Technology and Human Factors in Education and Health
