Technical Privacy Metrics: a Systematic Survey

TL;DR

This survey systematically reviews over eighty privacy metrics, categorizing them and providing a framework to select appropriate metrics for different scenarios to improve privacy measurement in digital systems.

Contribution

It offers a comprehensive structured overview of privacy metrics, introduces a method for selecting suitable metrics, and identifies gaps for future research.

Findings

Structured categorization of privacy metrics

A nine-question framework for metric selection

Identification of areas needing further metric development

Abstract

The goal of privacy metrics is to measure the degree of privacy enjoyed by users in a system and the amount of protection offered by privacy-enhancing technologies. In this way, privacy metrics contribute to improving user privacy in the digital world. The diversity and complexity of privacy metrics in the literature makes an informed choice of metrics challenging. As a result, instead of using existing metrics, new metrics are proposed frequently, and privacy studies are often incomparable. In this survey we alleviate these problems by structuring the landscape of privacy metrics. To this end, we explain and discuss a selection of over eighty privacy metrics and introduce categorizations based on the aspect of privacy they measure, their required inputs, and the type of data that needs protection. In addition, we present a method on how to choose privacy metrics based on nine questions that help identify the right privacy metrics for a given scenario, and highlight topics where additional work on privacy metrics is needed. Our survey spans multiple privacy domains and can be understood as a general framework for privacy measurement.