Will You Be Aware? Eye Tracking-Based Modeling of Situational Awareness in Augmented Reality

TL;DR

This study explores how eye tracking can model situational awareness in AR during CPR, revealing gaze patterns linked to awareness levels and introducing a neural network for prediction.

Contribution

It introduces FixGraphPool, a novel graph neural network that predicts situational awareness from eye tracking data in AR, outperforming existing models.

Findings

Higher SA linked to increased saccadic amplitude and velocity.

Reduced fixations on virtual content correlate with higher SA.

FixGraphPool achieved 83% accuracy in SA prediction.

Abstract

Augmented Reality (AR) systems, while enhancing task performance through real-time guidance, pose risks of inducing cognitive tunneling-a hyperfocus on virtual content that compromises situational awareness (SA) in safety-critical scenarios. This paper investigates SA in AR-guided cardiopulmonary resuscitation (CPR), where responders must balance effective compressions with vigilance to unpredictable hazards (e.g., patient vomiting). We developed an AR app on a Magic Leap 2 that overlays real-time CPR feedback (compression depth and rate) and conducted a user study with simulated unexpected incidents (e.g., bleeding) to evaluate SA, in which SA metrics were collected via observation and questionnaires administered during freeze-probe events. Eye tracking analysis revealed that higher SA levels were associated with greater saccadic amplitude and velocity, and with reduced proportion and frequency of fixations on virtual content. To predict SA, we propose FixGraphPool, a graph neural network that structures gaze events (fixations, saccades) into spatiotemporal graphs, effectively capturing dynamic attentional patterns. Our model achieved 83.0% accuracy (F1=81.0%), outperforming feature-based machine learning and state-of-the-art time-series models by leveraging domain knowledge and spatial-temporal information encoded in ET data. These findings demonstrate the potential of eye tracking for SA modeling in AR and highlight its utility in designing AR systems that ensure user safety and situational awareness.