Implicit Search Intent Recognition using EEG and Eye Tracking: Novel Dataset and Cross-User Prediction

TL;DR

This paper introduces a new EEG and eye-tracking dataset for recognizing visual search intents and proposes a cross-user prediction method achieving high accuracy, addressing real-world applicability issues in intent recognition.

Contribution

The paper provides the first publicly available dataset and a novel cross-user prediction approach for search intent recognition using EEG and eye-tracking data.

Findings

Achieved 84.5% accuracy in cross-user intent prediction

Comparable to within-user prediction accuracy of 85.5%

Addresses limitations of previous fixed-time and user-specific methods

Abstract

For machines to effectively assist humans in challenging visual search tasks, they must differentiate whether a human is simply glancing into a scene (navigational intent) or searching for a target object (informational intent). Previous research proposed combining electroencephalography (EEG) and eye-tracking measurements to recognize such search intents implicitly, i.e., without explicit user input. However, the applicability of these approaches to real-world scenarios suffers from two key limitations. First, previous work used fixed search times in the informational intent condition -- a stark contrast to visual search, which naturally terminates when the target is found. Second, methods incorporating EEG measurements addressed prediction scenarios that require ground truth training data from the target user, which is impractical in many use cases. We address these limitations by making the first publicly available EEG and eye-tracking dataset for navigational vs. informational intent recognition, where the user determines search times. We present the first method for cross-user prediction of search intents from EEG and eye-tracking recordings and reach 84.5% accuracy in leave-one-user-out evaluations -- comparable to within-user prediction accuracy (85.5%) but offering much greater flexibility