Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

TL;DR

This paper introduces a method to detect relevant information during decision-making from eye movements to enable adaptive user interfaces, demonstrating high accuracy and applicability across tasks without explicit user input.

Contribution

It presents a novel voting scheme combining six gaze metrics to robustly identify relevant information for UI adaptation, handling individual and task variability.

Findings

Detects up to 97% of relevant elements based on user self-report.

Outperforms standalone gaze metrics in identifying relevant information.

Enables real-time, task- and user-independent UI adaptation.

Abstract

This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, where gaze behavior is less complex. From the literature, we identify six metrics that capture different aspects of the gaze behavior during decision-making and combine them in a voting scheme. We empirically show, that this accounts for the large variations in gaze behavior and out-performs standalone metrics. Importantly, it offers an intuitive way to control the amount of detected information, which is crucial for different UI adaptation schemes to succeed. We show the applicability of our approach by developing a room-search application that changes the visual saliency of content detected as relevant. In an empirical study, we show that it detects up to 97% of relevant elements with respect to user self-reporting, which allows us to meaningfully adapt the interface, as confirmed by participants. Our approach is fast, does not need any explicit user input and can be applied independent of task and user.