Salient Object Detection for Images Taken by People With Vision Impairments

TL;DR

This paper introduces VizWiz-SalientObject, a large dataset of images taken by visually impaired individuals, to improve salient object detection, revealing current methods' limitations on such real-world, diverse images.

Contribution

The paper presents the largest dataset for salient object detection from visually impaired users' images, with unique features like high text prevalence and larger salient objects, and benchmarks existing methods on it.

Findings

Existing methods struggle with large, simple-boundary, low-text, low-quality images.

The dataset highlights the need for more robust salient object detection models.

Large, diverse dataset enables new research directions.

Abstract

Salient object detection is the task of producing a binary mask for an image that deciphers which pixels belong to the foreground object versus background. We introduce a new salient object detection dataset using images taken by people who are visually impaired who were seeking to better understand their surroundings, which we call VizWiz-SalientObject. Compared to seven existing datasets, VizWiz-SalientObject is the largest (i.e., 32,000 human-annotated images) and contains unique characteristics including a higher prevalence of text in the salient objects (i.e., in 68\% of images) and salient objects that occupy a larger ratio of the images (i.e., on average, $\sim$50\% coverage). We benchmarked seven modern salient object detection methods on our dataset and found they struggle most with images featuring salient objects that are large, have less complex boundaries, and lack text as well as for lower quality images. We invite the broader community to work on our new dataset challenge by publicly sharing the dataset at https://vizwiz.org/tasks-and-datasets/salient-object .