A Review of Privacy Metrics for Privacy-Preserving Synthetic Data Generation

TL;DR

This paper reviews and formalizes 17 privacy metrics used to evaluate privacy risks in synthetic data generation, aiming to clarify their assumptions and calculations for better assessment of privacy-preserving mechanisms.

Contribution

It provides a comprehensive analysis and formal definitions of 17 privacy metrics, addressing the lack of clarity and standardization in privacy risk assessment for synthetic data.

Findings

Formal definitions and assumptions of 17 privacy metrics

Highlights the diversity and complexity of privacy metrics

Facilitates better comparison and understanding of privacy risks

Abstract

Privacy Preserving Synthetic Data Generation (PP-SDG) has emerged to produce synthetic datasets from personal data while maintaining privacy and utility. Differential privacy (DP) is the property of a PP-SDG mechanism that establishes how protected individuals are when sharing their sensitive data. It is however difficult to interpret the privacy budget ($\varepsilon$) expressed by DP. To make the actual risk associated with the privacy budget more transparent, multiple privacy metrics (PMs) have been proposed to assess the privacy risk of the data. These PMs are utilized in separate studies to assess newly introduced PP-SDG mechanisms. Consequently, these PMs embody the same assumptions as the PP-SDG mechanism they were made to assess. Therefore, a thorough definition of how these are calculated is necessary. In this work, we present the assumptions and mathematical formulations of 17 distinct privacy metrics.