Relative Saliency and Ranking: Models, Metrics, Data, and Benchmarks

TL;DR

This paper introduces a new deep learning approach for relative saliency ranking of objects, along with datasets, metrics, and benchmarks to evaluate and improve salient object detection and ranking.

Contribution

It proposes a hierarchical deep learning model for relative saliency, introduces datasets and metrics for ranking, and establishes baseline benchmarks for future research.

Findings

A novel deep learning model for relative saliency ranking.

A dataset and metrics for evaluating salient object ranking.

Baseline benchmark results for future comparison.

Abstract

Salient object detection is a problem that has been considered in detail and \textcolor{black}{many solutions have been proposed}. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that some objects are more likely to be judged salient than others, and implies a relative rank exists on salient objects. Initially, we present a novel deep learning solution based on a hierarchical representation of relative saliency and stage-wise refinement. Further to this, we present data, analysis and baseline benchmark results towards addressing the problem of salient object ranking. Methods for deriving suitable ranked salient object instances are presented, along with metrics suitable to measuring algorithm performance. In addition, we show how a derived dataset can be successively refined to provide cleaned results that correlate well with pristine ground truth in its characteristics and value for training and testing models. Finally, we provide a comparison among prevailing algorithms that address salient object ranking or detection to establish initial baselines providing a basis for comparison with future efforts addressing this problem. \textcolor{black}{The source code and data are publicly available via our project page:} \textrm{\href{https://ryersonvisionlab.github.io/cocosalrank.html}{ryersonvisionlab.github.io/cocosalrank}}