Edgeworth Accountant: An Analytical Approach to Differential Privacy Composition

TL;DR

The paper introduces the Edgeworth Accountant, an analytical method leveraging the Edgeworth expansion to accurately and efficiently compute differential privacy guarantees under composition, applicable to various noise-addition mechanisms.

Contribution

It presents a novel analytical approach using the Edgeworth expansion to track privacy loss, providing tight, non-asymptotic $(, )$-DP bounds with low computational overhead.

Findings

Provides a closed-form expression for privacy guarantees.

Applicable to any noise-addition mechanism.

Offers accurate estimates and tight bounds for deep learning and federated analytics.

Abstract

In privacy-preserving data analysis, many procedures and algorithms are structured as compositions of multiple private building blocks. As such, an important question is how to efficiently compute the overall privacy loss under composition. This paper introduces the Edgeworth Accountant, an analytical approach to composing differential privacy guarantees for private algorithms. Leveraging the $f$-differential privacy framework, the Edgeworth Accountant accurately tracks privacy loss under composition, enabling a closed-form expression of privacy guarantees through privacy-loss log-likelihood ratios (PLLRs). As implied by its name, this method applies the Edgeworth expansion to estimate and define the probability distribution of the sum of the PLLRs. Furthermore, by using a technique that simplifies complex distributions into simpler ones, we demonstrate the Edgeworth Accountant's applicability to any noise-addition mechanism. Its main advantage is providing $(\epsilon, \delta)$-differential privacy bounds that are non-asymptotic and do not significantly increase computational cost. This feature sets it apart from previous approaches, in which the running time increases with the number of mechanisms under composition. We conclude by showing how our Edgeworth Accountant offers accurate estimates and tight upper and lower bounds on $(\epsilon, \delta)$-differential privacy guarantees, especially tailored for training private models in deep learning and federated analytics.