Salient Object Detection in the Deep Learning Era: An In-Depth Survey

TL;DR

This paper provides a comprehensive survey of deep learning-based salient object detection, analyzing algorithms, datasets, evaluation metrics, and robustness, while also introducing a new dataset and discussing future challenges.

Contribution

It offers an in-depth taxonomy of deep SOD methods, introduces a novel attribute-rich dataset, and explores robustness and generalization issues in the field.

Findings

Benchmarking results of various SOD models

Analysis of model robustness to perturbations and attacks

Insights into dataset difficulty and generalization challenges

Abstract

As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years. Recent advances in SOD are predominantly led by deep learning-based solutions (named deep SOD). To enable in-depth understanding of deep SOD, in this paper, we provide a comprehensive survey covering various aspects, ranging from algorithm taxonomy to unsolved issues. In particular, we first review deep SOD algorithms from different perspectives, including network architecture, level of supervision, learning paradigm, and object-/instance-level detection. Following that, we summarize and analyze existing SOD datasets and evaluation metrics. Then, we benchmark a large group of representative SOD models, and provide detailed analyses of the comparison results. Moreover, we study the performance of SOD algorithms under different attribute settings, which has not been thoroughly explored previously, by constructing a novel SOD dataset with rich attribute annotations covering various salient object types, challenging factors, and scene categories. We further analyze, for the first time in the field, the robustness of SOD models to random input perturbations and adversarial attacks. We also look into the generalization and difficulty of existing SOD datasets. Finally, we discuss several open issues of SOD and outline future research directions.