On Improving the Composition Privacy Loss in Differential Privacy for Fixed Estimation Error

TL;DR

This paper introduces an iterative suppression-based algorithm to improve privacy loss in differential privacy when releasing subset statistics, maintaining fixed estimation error and addressing data heterogeneity.

Contribution

It proposes a novel iterative method that reduces privacy loss degradation without increasing estimation error, with analytical sensitivity and bias characterizations.

Findings

Demonstrates improved privacy loss in real-world datasets

Maintains fixed worst-case estimation error

Provides analytical sensitivity and bias analysis

Abstract

This paper considers the private release of statistics of disjoint subsets of a dataset, in the setting of data heterogeneity, where users could contribute more than one sample, with different users contributing potentially different numbers of samples. In particular, we focus on the $\epsilon$-differentially private release of sample means and variances of sample values in disjoint subsets of a dataset, under the assumption that the numbers of contributions of each user in each subset is publicly known. Our main contribution is an iterative algorithm, based on suppressing user contributions, which seeks to reduce the overall privacy loss degradation under a canonical Laplace mechanism, while not increasing the worst estimation error among the subsets. Important components of this analysis are our exact, analytical characterizations of the sensitivities and the worst-case bias errors of estimators of the sample mean and variance, which are obtained by clipping or suppressing user contributions. We test the performance of our algorithm on real-world and synthetic datasets and demonstrate clear improvements in the privacy loss degradation, for fixed worst-case estimation error.