Learning Efficient Representations of Mouse Movements to Predict User Attention

TL;DR

This paper explores various neural network models trained on raw mouse cursor data to predict user attention on web pages, offering a more efficient alternative to handcrafted features for user behavior analysis.

Contribution

It introduces novel representations of mouse movements and demonstrates the effectiveness of neural networks in predicting user attention, reducing reliance on manual feature engineering.

Findings

Neural networks trained on raw cursor data achieve competitive accuracy.

Different representations like heatmaps and trajectories improve model performance.

Models can be used for implicit feedback in re-ranking and evaluation tasks.

Abstract

Tracking mouse cursor movements can be used to predict user attention on heterogeneous page layouts like SERPs. So far, previous work has relied heavily on handcrafted features, which is a time-consuming approach that often requires domain expertise. We investigate different representations of mouse cursor movements, including time series, heatmaps, and trajectory-based images, to build and contrast both recurrent and convolutional neural networks that can predict user attention to direct displays, such as SERP advertisements. Our models are trained over raw mouse cursor data and achieve competitive performance. We conclude that neural network models should be adopted for downstream tasks involving mouse cursor movements, since they can provide an invaluable implicit feedback signal for re-ranking and evaluation.