Semantics Meet Saliency: Exploring Domain Affinity and Models for Dual-Task Prediction

TL;DR

This paper investigates the relationship between semantic labeling and saliency prediction in images, proposing dual-task neural networks and analyzing dataset biases and semantic precedence for improved understanding.

Contribution

It introduces deep neural networks capable of jointly predicting semantic labels and salient regions, and provides a detailed analysis of dataset biases and semantic relationships.

Findings

Joint prediction improves understanding of scene composition.

Semantic precedence influences saliency detection.

Dataset biases affect model performance and interpretation.

Abstract

Much research has examined models for prediction of semantic labels or instances including dense pixel-wise prediction. The problem of predicting salient objects or regions of an image has also been examined in a similar light. With that said, there is an apparent relationship between these two problem domains in that the composition of a scene and associated semantic categories is certain to play into what is deemed salient. In this paper, we explore the relationship between these two problem domains. This is carried out in constructing deep neural networks that perform both predictions together albeit with different configurations for flow of conceptual information related to each distinct problem. This is accompanied by a detailed analysis of object co-occurrences that shed light on dataset bias and semantic precedence specific to individual categories.