Prototyping and Evaluating a Real-time Neuro-Adaptive Virtual Reality Flight Training System

TL;DR

This study developed and tested a neuro-adaptive VR flight training system that adjusts difficulty based on brain signals, comparing it to fixed difficulty sequences in pilot training.

Contribution

It introduces a pre-trained EEG-based BCI system for real-time difficulty adaptation in VR flight training, assessing its effectiveness and pilot perceptions.

Findings

No significant difference in performance or subjective measures between adaptive and fixed conditions.

Flight performance decreased with increased subjective workload in both conditions.

Pilots preferred the neuro-adaptive system despite similar performance outcomes.

Abstract

Real-time adjustments to task difficulty during flight training are crucial for optimizing performance and managing pilot workload. This study evaluated the functionality of a pre-trained brain-computer interface (BCI) that adapts training difficulty based on real-time estimations of workload from brain signals. Specifically, an EEG-based neuro-adaptive training system was developed and tested in Virtual Reality (VR) flight simulations with military student pilots. The neuro-adaptive system was compared to a fixed sequence that progressively increased in difficulty, in terms of self-reported user engagement, workload, and simulator sickness (subjective measures), as well as flight performance (objective metric). Additionally, we explored the relationships between subjective workload and flight performance in the VR simulator for each condition. The experiments concluded with semi-structured interviews to elicit the pilots' experience with the neuro-adaptive prototype. Results revealed no significant differences between the adaptive and fixed sequence conditions in subjective measures or flight performance. In both conditions, flight performance decreased as subjective workload increased. The semi-structured interviews indicated that, upon briefing, the pilots preferred the neuro-adaptive VR training system over the system with a fixed sequence, although individual differences were observed in the perception of difficulty and the order of changes in difficulty. Even though this study shows performance does not change, BCI-based flight training systems hold the potential to provide a more personalized and varied training experience.