On the Relationship Between Information-Theoretic Privacy Metrics And Probabilistic Information Privacy

TL;DR

This paper connects information-theoretic privacy metrics with probabilistic information privacy, providing operational interpretations and developing a neural network-based empirical risk framework for privacy preservation.

Contribution

It relates IT privacy metrics to probabilistic IP, interprets them under detection and estimation, and introduces a neural network framework based on the $ ext{chi}^2$-divergence for practical privacy protection.

Findings

$ ext{chi}^2$-divergence privacy is stronger than total variation and KL divergence.

The neural network framework effectively enforces privacy in experiments.

The approach is agnostic to adversarial attack models.

Abstract

Information-theoretic (IT) measures based on $f$-divergences have recently gained interest as a measure of privacy leakage as they allow for trading off privacy against utility using only a single-value characterization. However, their operational interpretations in the privacy context are unclear. In this paper, we relate the notion of probabilistic information privacy (IP) to several IT privacy metrics based on $f$-divergences. We interpret probabilistic IP under both the detection and estimation frameworks and link it to differential privacy, thus allowing a precise operational interpretation of these IT privacy metrics. We show that the $\chi^2$-divergence privacy metric is stronger than those based on total variation distance and Kullback-Leibler divergence. Therefore, we further develop a data-driven empirical risk framework based on the $\chi^2$-divergence privacy metric and realized using deep neural networks. This framework is agnostic to the adversarial attack model. Empirical experiments demonstrate the efficacy of our approach.