Saliency Integration: An Arbitrator Model

TL;DR

This paper introduces an arbitrator model for saliency integration that combines multiple models and external knowledge, estimating model expertise without ground truth to improve saliency map accuracy.

Contribution

The proposed arbitrator model effectively integrates multiple saliency models using a Bayesian framework and online expertise estimation, addressing previous challenges.

Findings

Significant performance improvement over existing methods

Effective integration across traditional and deep learning models

Robustness across multiple datasets

Abstract

Saliency integration has attracted much attention on unifying saliency maps from multiple saliency models. Previous offline integration methods usually face two challenges: 1. if most of the candidate saliency models misjudge the saliency on an image, the integration result will lean heavily on those inferior candidate models; 2. an unawareness of the ground truth saliency labels brings difficulty in estimating the expertise of each candidate model. To address these problems, in this paper, we propose an arbitrator model (AM) for saliency integration. Firstly, we incorporate the consensus of multiple saliency models and the external knowledge into a reference map to effectively rectify the misleading by candidate models. Secondly, our quest for ways of estimating the expertise of the saliency models without ground truth labels gives rise to two distinct online model-expertise estimation methods. Finally, we derive a Bayesian integration framework to reconcile the saliency models of varying expertise and the reference map. To extensively evaluate the proposed AM model, we test twenty-seven state-of-the-art saliency models, covering both traditional and deep learning ones, on various combinations over four datasets. The evaluation results show that the AM model improves the performance substantially compared to the existing state-of-the-art integration methods, regardless of the chosen candidate saliency models.