Clustered Saliency Prediction

TL;DR

This paper introduces a novel clustered saliency prediction method that personalizes image salience models based on subject clusters, improving prediction accuracy over universal models.

Contribution

It proposes a clustering-based approach for personalized saliency prediction and a Multi-Domain Saliency Translation model utilizing state-of-the-art saliency methods.

Findings

Clustered saliency prediction outperforms universal models.

Clustering improves personalization accuracy.

Effective method for assigning new subjects to clusters.

Abstract

We present a new method for image salience prediction, Clustered Saliency Prediction. This method divides subjects into clusters based on their personal features and their known saliency maps, and generates an image salience model conditioned on the cluster label. We test our approach on a public dataset of personalized saliency maps and cluster the subjects using selected importance weights for personal feature factors. We propose the Multi-Domain Saliency Translation model which uses image stimuli and universal saliency maps to predict saliency maps for each cluster. For obtaining universal saliency maps, we applied various state-of-the-art methods, DeepGaze IIE, ML-Net and SalGAN, and compared their effectiveness in our system. We show that our Clustered Saliency Prediction technique outperforms the universal saliency prediction models. Also, we demonstrate the effectiveness of our clustering method by comparing the results of Clustered Saliency Prediction using clusters obtained by our algorithm with some baseline methods. Finally, we propose an approach to assign new people to their most appropriate cluster and prove its usefulness in the experiments.