Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction

TL;DR

This study investigates how visual highlighting and cognitive load influence user attention and saliency prediction, revealing that dynamic highlights remain effective under load and improving saliency models' performance.

Contribution

It provides empirical evidence on the effects of highlighting and cognitive load on attention, and introduces a dataset to improve saliency prediction models under multitasking conditions.

Findings

Dynamic highlighting remains attention-grabbing under cognitive load.

Saliency models perform better when accounting for cognitive load.

Cognitive load significantly alters attention and saliency patterns.

Abstract

Visual highlighting can guide user attention in complex interfaces. However, its effectiveness under limited attentional capacities is underexplored. This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. Our analysis, using eye-movement data from 27 participants viewing 150 unique webpages reveals that while participants' ability to attend to UI elements decreases with increasing cognitive load, dynamic adaptations (i.e., highlighting) remain attention-grabbing. The presence of these factors significantly alters what people attend to and thus what is salient. Accordingly, we show that state-of-the-art saliency models increase their performance when accounting for different cognitive loads. Our empirical insights, along with our openly available dataset, enhance our understanding of attentional processes in UIs under varying cognitive (and perceptual) loads and open the door for new models that can predict user attention while multitasking.