Pluralistic Salient Object Detection

TL;DR

This paper introduces a new task called pluralistic salient object detection (PSOD) that generates multiple plausible segmentation masks for images, addressing ambiguity and multiple object saliencies, supported by new datasets and a Mixture-of-Experts model.

Contribution

The paper proposes the first pluralistic SOD framework, new datasets with enhanced annotations and human preferences, and demonstrates its effectiveness through extensive experiments.

Findings

Datasets improve boundary and fine-grained mask quality.

The MOE-based model predicts multiple masks and human preferences.

Experimental results validate the proposed approach's effectiveness.

Abstract

We introduce pluralistic salient object detection (PSOD), a novel task aimed at generating multiple plausible salient segmentation results for a given input image. Unlike conventional SOD methods that produce a single segmentation mask for salient objects, this new setting recognizes the inherent complexity of real-world images, comprising multiple objects, and the ambiguity in defining salient objects due to different user intentions. To study this task, we present two new SOD datasets "DUTS-MM" and "DUS-MQ", along with newly designed evaluation metrics. DUTS-MM builds upon the DUTS dataset but enriches the ground-truth mask annotations from three aspects which 1) improves the mask quality especially for boundary and fine-grained structures; 2) alleviates the annotation inconsistency issue; and 3) provides multiple ground-truth masks for images with saliency ambiguity. DUTS-MQ consists of approximately 100K image-mask pairs with human-annotated preference scores, enabling the learning of real human preferences in measuring mask quality. Building upon these two datasets, we propose a simple yet effective pluralistic SOD baseline based on a Mixture-of-Experts (MOE) design. Equipped with two prediction heads, it simultaneously predicts multiple masks using different query prompts and predicts human preference scores for each mask candidate. Extensive experiments and analyses underscore the significance of our proposed datasets and affirm the effectiveness of our PSOD framework.