Auditing Privacy Mechanisms via Label Inference Attacks

TL;DR

This paper introduces reconstruction advantage measures to evaluate how well privacy mechanisms protect label information, enabling comparison of various schemes including differentially private methods, through theoretical analysis and empirical experiments.

Contribution

It proposes new quantitative measures for auditing label privacy mechanisms, unifying evaluation of private and non-private schemes under a common framework.

Findings

Differentially private schemes outperform heuristic approaches in privacy-utility tradeoff.

Reconstruction advantage measures effectively quantify label inference risks.

Theoretical analysis supports empirical results across multiple datasets.

Abstract

We propose reconstruction advantage measures to audit label privatization mechanisms. A reconstruction advantage measure quantifies the increase in an attacker's ability to infer the true label of an unlabeled example when provided with a private version of the labels in a dataset (e.g., aggregate of labels from different users or noisy labels output by randomized response), compared to an attacker that only observes the feature vectors, but may have prior knowledge of the correlation between features and labels. We consider two such auditing measures: one additive, and one multiplicative. These incorporate previous approaches taken in the literature on empirical auditing and differential privacy. The measures allow us to place a variety of proposed privatization schemes -- some differentially private, some not -- on the same footing. We analyze these measures theoretically under a distributional model which encapsulates reasonable adversarial settings. We also quantify their behavior empirically on real and simulated prediction tasks. Across a range of experimental settings, we find that differentially private schemes dominate or match the privacy-utility tradeoff of more heuristic approaches.