Real or Virtual? Using Brain Activity Patterns to differentiate Attended Targets during Augmented Reality Scenarios

TL;DR

This study demonstrates that EEG-based machine learning can reliably distinguish whether a user is attending to real or virtual objects in augmented reality, suggesting potential for brain-computer interfaces in AR.

Contribution

It introduces a novel approach using EEG and deep learning to classify attention to real versus virtual targets in AR scenarios, with promising accuracy.

Findings

Person-dependent classification accuracy above 70%

Person-independent classification above chance for some participants

EEG-based classification shows potential for AR input mechanisms

Abstract

Augmented Reality is the fusion of virtual components and our real surroundings. The simultaneous visibility of generated and natural objects often requires users to direct their selective attention to a specific target that is either real or virtual. In this study, we investigated whether this target is real or virtual by using machine learning techniques to classify electroencephalographic (EEG) data collected in Augmented Reality scenarios. A shallow convolutional neural net classified 3 second data windows from 20 participants in a person-dependent manner with an average accuracy above 70\% if the testing data and training data came from different trials. Person-independent classification was possible above chance level for 6 out of 20 participants. Thus, the reliability of such a Brain-Computer Interface is high enough for it to be treated as a useful input mechanism for Augmented Reality applications.