Rethinking of the Image Salient Object Detection: Object-level Semantic Saliency Re-ranking First, Pixel-wise Saliency Refinement Latter

TL;DR

This paper proposes a two-stage approach to salient object detection that first identifies semantically salient regions using a lightweight network and then refines pixel-wise saliency through multi-model fusion, aligning more closely with human attention.

Contribution

It introduces a novel object-level semantic re-ranking framework for SOD, focusing on semantic saliency across images rather than solely pixel-wise features within a single image.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effectively captures semantic saliency aligned with human attention.

Demonstrates the effectiveness of object-level re-ranking in SOD.

Abstract

The real human attention is an interactive activity between our visual system and our brain, using both low-level visual stimulus and high-level semantic information. Previous image salient object detection (SOD) works conduct their saliency predictions in a multi-task manner, i.e., performing pixel-wise saliency regression and segmentation-like saliency refinement at the same time, which degenerates their feature backbones in revealing semantic information. However, given an image, we tend to pay more attention to those regions which are semantically salient even in the case that these regions are perceptually not the most salient ones at first glance. In this paper, we divide the SOD problem into two sequential tasks: 1) we propose a lightweight, weakly supervised deep network to coarsely locate those semantically salient regions first; 2) then, as a post-processing procedure, we selectively fuse multiple off-the-shelf deep models on these semantically salient regions as the pixel-wise saliency refinement. In sharp contrast to the state-of-the-art (SOTA) methods that focus on learning pixel-wise saliency in "single image" using perceptual clues mainly, our method has investigated the "object-level semantic ranks between multiple images", of which the methodology is more consistent with the real human attention mechanism. Our method is simple yet effective, which is the first attempt to consider the salient object detection mainly as an object-level semantic re-ranking problem.