Semantic and Contrast-Aware Saliency

TL;DR

This paper introduces a combined semantic and contrast-aware saliency model that integrates bottom-up and top-down cues for improved eye fixation prediction, demonstrating superior performance on multiple datasets.

Contribution

It presents a novel integrated saliency model with dual pathways, including deep neural network and feature integration implementations, enhancing saliency estimation accuracy.

Findings

Outperforms classic and deep models on benchmark datasets.

Effectively captures meaningful objects and high contrast patterns.

Demonstrates superior plausibility and robustness.

Abstract

In this paper, we proposed an integrated model of semantic-aware and contrast-aware saliency combining both bottom-up and top-down cues for effective saliency estimation and eye fixation prediction. The proposed model processes visual information using two pathways. The first pathway aims to capture the attractive semantic information in images, especially for the presence of meaningful objects and object parts such as human faces. The second pathway is based on multi-scale on-line feature learning and information maximization, which learns an adaptive sparse representation for the input and discovers the high contrast salient patterns within the image context. The two pathways characterize both long-term and short-term attention cues and are integrated dynamically using maxima normalization. We investigate two different implementations of the semantic pathway including an End-to-End deep neural network solution and a dynamic feature integration solution, resulting in the SCA and SCAFI model respectively. Experimental results on artificial images and 5 popular benchmark datasets demonstrate the superior performance and better plausibility of the proposed model over both classic approaches and recent deep models.