Learning Numeric Optimal Differentially Private Truncated Additive Mechanisms

TL;DR

This paper introduces a gradient-descent-based method to learn truncated noise mechanisms for differential privacy, achieving strong utility bounds and improved privacy-utility trade-offs, especially in sequential query scenarios.

Contribution

It presents a novel approach to optimize truncated additive noise mechanisms for differential privacy, including discrete patterns and extending Moments Accountant to truncated distributions.

Findings

Utility-privacy trade-off curves closely match truncated Gaussians.

Improved DP-SGD performance with sub-sampling for low composition counts.

Extended Moments Accountant to handle truncated distributions.

Abstract

Differentially private (DP) mechanisms face the challenge of providing accurate results while protecting their inputs: the privacy-utility trade-off. A simple but powerful technique for DP adds noise to sensitivity-bounded query outputs to blur the exact query output: additive mechanisms. While a vast body of work considers infinitely wide noise distributions, some applications (e.g., real-time operating systems) require hard bounds on the deviations from the real query, and only limited work on such mechanisms exist. An additive mechanism with truncated noise (i.e., with bounded range) can offer such hard bounds. We introduce a gradient-descent-based tool to learn truncated noise for additive mechanisms with strong utility bounds while simultaneously optimizing for differential privacy under sequential composition, i.e., scenarios where multiple noisy queries on the same data are revealed. Our method can learn discrete noise patterns and not only hyper-parameters of a predefined probability distribution. For sensitivity bounded mechanisms, we show that it is sufficient to consider symmetric and that\new{, for from the mean monotonically falling noise,} ensuring privacy for a pair of representative query outputs guarantees privacy for all pairs of inputs (that differ in one element). We find that the utility-privacy trade-off curves of our generated noise are remarkably close to truncated Gaussians and even replicate their shape for $l_2$ utility-loss. For a low number of compositions, we also improved DP-SGD (sub-sampling). Moreover, we extend Moments Accountant to truncated distributions, allowing to incorporate mechanism output events with varying input-dependent zero occurrence probability.