BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments

TL;DR

This paper introduces BIV-Priv-Seg, a new dataset with segmentation annotations for private content in images taken by visually impaired individuals, highlighting challenges faced by current models in locating non-salient private objects.

Contribution

The paper presents the first dataset of its kind for private content localization in images from visually impaired users and evaluates model performance on this challenging task.

Findings

Modern models struggle with locating non-salient private objects.

Models have difficulty recognizing absence of private content.

Small and text-lacking private objects are particularly challenging.

Abstract

Individuals who are blind or have low vision (BLV) are at a heightened risk of sharing private information if they share photographs they have taken. To facilitate developing technologies that can help them preserve privacy, we introduce BIV-Priv-Seg, the first localization dataset originating from people with visual impairments that shows private content. It contains 1,028 images with segmentation annotations for 16 private object categories. We first characterize BIV-Priv-Seg and then evaluate modern models' performance for locating private content in the dataset. We find modern models struggle most with locating private objects that are not salient, small, and lack text as well as recognizing when private content is absent from an image. We facilitate future extensions by sharing our new dataset with the evaluation server at https://vizwiz.org/tasks-and-datasets/object-localization.