Personalization of Saliency Estimation

TL;DR

This paper introduces a personalized saliency prediction model that incorporates individual observer traits, improving attention prediction accuracy over traditional models that use only low-level features or task descriptions.

Contribution

The paper presents a novel GAN-based model that personalizes saliency maps by integrating observer identity and traits, advancing beyond average observer models.

Findings

Improved prediction accuracy for personalized saliency maps.

Model effectively incorporates observer traits into saliency prediction.

Outperforms benchmark models in diverse observer groups.

Abstract

Most existing saliency models use low-level features or task descriptions when generating attention predictions. However, the link between observer characteristics and gaze patterns is rarely investigated. We present a novel saliency prediction technique which takes viewers' identities and personal traits into consideration when modeling human attention. Instead of only computing image salience for average observers, we consider the interpersonal variation in the viewing behaviors of observers with different personal traits and backgrounds. We present an enriched derivative of the GAN network, which is able to generate personalized saliency predictions when fed with image stimuli and specific information about the observer. Our model contains a generator which generates grayscale saliency heat maps based on the image and an observer label. The generator is paired with an adversarial discriminator which learns to distinguish generated salience from ground truth salience. The discriminator also has the observer label as an input, which contributes to the personalization ability of our approach. We evaluate the performance of our personalized salience model by comparison with a benchmark model along with other un-personalized predictions, and illustrate improvements in prediction accuracy for all tested observer groups.