Personalized Privacy Amplification via Importance Sampling

TL;DR

This paper explores how importance sampling affects privacy in scalable machine learning, proposing methods to optimize privacy-utility trade-offs and demonstrating improved performance over uniform sampling in differentially private k-means clustering.

Contribution

It introduces personalized privacy amplification techniques via importance sampling, balancing privacy and utility, with empirical evaluation on differentially private clustering.

Findings

Importance sampling aligns privacy with utility but reduces sample size.

Proposed sampling methods improve privacy-utility trade-off.

Both methods outperform uniform sampling in experiments.

Abstract

For scalable machine learning on large data sets, subsampling a representative subset is a common approach for efficient model training. This is often achieved through importance sampling, whereby informative data points are sampled more frequently. In this paper, we examine the privacy properties of importance sampling, focusing on an individualized privacy analysis. We find that, in importance sampling, privacy is well aligned with utility but at odds with sample size. Based on this insight, we propose two approaches for constructing sampling distributions: one that optimizes the privacy-efficiency trade-off; and one based on a utility guarantee in the form of coresets. We evaluate both approaches empirically in terms of privacy, efficiency, and accuracy on the differentially private $k$-means problem. We observe that both approaches yield similar outcomes and consistently outperform uniform sampling across a wide range of data sets. Our code is available on GitHub: https://github.com/smair/personalized-privacy-amplification-via-importance-sampling