Beyond Blanket Masking: Examining Granularity for Privacy Protection in Images Captured by Blind and Low Vision Users

TL;DR

This paper introduces FiGPriv, a fine-grained privacy protection framework for images captured by blind and low vision users, selectively masking private information to balance privacy and usability.

Contribution

It presents a novel fine-grained segmentation and risk scoring approach for privacy protection in images, improving content preservation and VLM utility.

Findings

Preserves +26% of image content compared to coarse masking

Enhances VLM response usefulness by 11%

Improves image content identification by 45%

Abstract

As visual assistant systems powered by visual language models (VLMs) become more prevalent, concerns over user privacy have grown, particularly for blind and low vision users who may unknowingly capture personal private information in their images. Existing privacy protection methods rely on coarse-grained segmentation, which uniformly masks entire private objects, often at the cost of usability. In this work, we propose FiGPriv, a fine-grained privacy protection framework that selectively masks only high-risk private information while preserving low-risk information. Our approach integrates fine-grained segmentation with a data-driven risk scoring mechanism. We evaluate our framework using the BIV-Priv-Seg dataset and show that FiG-Priv preserves +26% of image content, enhancing the ability of VLMs to provide useful responses by 11% and identify the image content by 45%, while ensuring privacy protection. Project Page: https://artcs1.github.io/VLMPrivacy/