Evaluation of Human Visual Privacy Protection: A Three-Dimensional Framework and Benchmark Dataset

TL;DR

This paper introduces a three-dimensional framework for evaluating visual privacy protection methods, along with a new dataset, enabling objective assessment of privacy, utility, and practicality in AI surveillance systems.

Contribution

It proposes a novel comprehensive evaluation framework and provides the HR-VISPR dataset for assessing privacy protection techniques in human-centric visual data.

Findings

Differentiates privacy levels aligned with human perception

Highlights trade-offs between privacy, utility, and practicality

Evaluates 11 privacy protection methods using the framework

Abstract

Recent advances in AI-powered surveillance have intensified concerns over the collection and processing of sensitive personal data. In response, research has increasingly focused on privacy-by-design solutions, raising the need for objective techniques to evaluate privacy protection. This paper presents a comprehensive framework for evaluating visual privacy-protection methods across three dimensions: privacy, utility, and practicality. In addition, it introduces HR-VISPR, a publicly available human-centric dataset with biometric, soft-biometric, and non-biometric labels to train an interpretable privacy metric. We evaluate 11 privacy protection methods, ranging from conventional techniques to advanced deep-learning methods, through the proposed framework. The framework differentiates privacy levels in alignment with human visual perception, while highlighting trade-offs between privacy, utility, and practicality. This study, along with the HR-VISPR dataset, serves as an insightful tool and offers a structured evaluation framework applicable across diverse contexts.