Impact of Sampling on Locally Differentially Private Data Collection

TL;DR

This paper examines how sampling affects the utility of locally differentially private data collection, proposing new unbiased estimators for frequency estimation under sampling in distributed environments like federated learning.

Contribution

It introduces a novel unbiased estimator for frequency estimation with sampling, and explores personalized sampling probabilities to improve privacy and utility trade-offs.

Findings

Sampling impacts estimator bias and utility.

Proposed estimators maintain unbiasedness under sampling.

Trade-offs between communication cost, privacy, and utility are analyzed.

Abstract

With the recent bloom of data, there is a huge surge in threats against individuals' private information. Various techniques for optimizing privacy-preserving data analysis are at the focus of research in the recent years. In this paper, we analyse the impact of sampling on the utility of the standard techniques of frequency estimation, which is at the core of large-scale data analysis, of the locally deferentially private data-release under a pure protocol. We study the case in a distributed environment of data sharing where the values are reported by various nodes to the central server, e.g., cross-device Federated Learning. We show that if we introduce some random sampling of the nodes in order to reduce the cost of communication, the standard existing estimators fail to remain unbiased. We propose a new unbiased estimator in the context of sampling each node with certain probability and compute various statistical summaries of the data using it. We propose a way of sampling each node with personalized sampling probabilities as a step to further generalisation, which leads to some interesting open questions in the end. We analyse the accuracy of our proposed estimators on synthetic datasets to gather some insight on the trade-off between communication cost, privacy, and utility.