Saliency Prediction for Mobile User Interfaces

TL;DR

This paper presents a novel deep learning model for predicting visual saliency in mobile user interfaces, focusing on interface elements rather than natural images, and demonstrates improved performance over existing methods.

Contribution

The paper introduces a new autoencoder-based multi-scale deep learning model specifically designed for saliency prediction in mobile interfaces, incorporating element-level analysis.

Findings

Model outperforms existing natural image saliency methods on mobile interface data

Eye-gaze data collected from mobile devices enables effective training of the model

Saliency prediction at element level improves understanding of user attention in mobile UI

Abstract

We introduce models for saliency prediction for mobile user interfaces. A mobile interface may include elements like buttons, text, etc. in addition to natural images which enable performing a variety of tasks. Saliency in natural images is a well studied area. However, given the difference in what constitutes a mobile interface, and the usage context of these devices, we postulate that saliency prediction for mobile interface images requires a fresh approach. Mobile interface design involves operating on elements, the building blocks of the interface. We first collected eye-gaze data from mobile devices for free viewing task. Using this data, we develop a novel autoencoder based multi-scale deep learning model that provides saliency prediction at the mobile interface element level. Compared to saliency prediction approaches developed for natural images, we show that our approach performs significantly better on a range of established metrics.