Quantifying Membership Privacy via Information Leakage

TL;DR

This paper introduces a method to quantify and analyze the privacy risks of machine learning models, especially regarding membership inference, using information leakage measures, and applies it to the PATE framework.

Contribution

It proposes a novel information leakage-based approach for membership privacy, analyzing the PATE framework and deriving bounds for privacy leakage with Laplace noise.

Findings

Entrywise information leakage is Schur-concave with log-concave noise.

Increased teacher consensus reduces privacy cost.

Upper bounds on leakage are derived for Laplace noise.

Abstract

Machine learning models are known to memorize the unique properties of individual data points in a training set. This memorization capability can be exploited by several types of attacks to infer information about the training data, most notably, membership inference attacks. In this paper, we propose an approach based on information leakage for guaranteeing membership privacy. Specifically, we propose to use a conditional form of the notion of maximal leakage to quantify the information leaking about individual data entries in a dataset, i.e., the entrywise information leakage. We apply our privacy analysis to the Private Aggregation of Teacher Ensembles (PATE) framework for privacy-preserving classification of sensitive data and prove that the entrywise information leakage of its aggregation mechanism is Schur-concave when the injected noise has a log-concave probability density. The Schur-concavity of this leakage implies that increased consensus among teachers in labeling a query reduces its associated privacy cost. Finally, we derive upper bounds on the entrywise information leakage when the aggregation mechanism uses Laplace distributed noise.