Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

TL;DR

This paper introduces a black-box auditing method for DP-SGD that provides tighter privacy estimates than previous approaches, helping to identify privacy violations and improve analysis in real-world machine learning models.

Contribution

It proposes a novel worst-case initial model parameter crafting technique for black-box privacy auditing of DP-SGD, achieving significantly tighter privacy bounds.

Findings

Empirical privacy estimates are substantially lower than theoretical bounds.

Auditing method is effective on MNIST and CIFAR-10 datasets.

Detects privacy violations in real-world implementations.

Abstract

This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical $\varepsilon=10.0$, our auditing procedure yields empirical estimates of $\varepsilon_{emp} = 7.21$ and $6.95$, respectively, on a 1,000-record sample and $\varepsilon_{emp}= 6.48$ and $4.96$ on the full datasets. By contrast, previous audits were only (relatively) tight in stronger white-box models, where the adversary can access the model's inner parameters and insert arbitrary gradients. Overall, our auditing procedure can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations. The source code needed to reproduce our experiments is available at https://github.com/spalabucr/bb-audit-dpsgd.