Auditing Private Prediction
Karan Chadha, Matthew Jagielski, Nicolas Papernot, Christopher, Choquette-Choo, Milad Nasr

TL;DR
This paper introduces a novel framework for empirically auditing the privacy leakage of private prediction algorithms, revealing that some algorithms are more vulnerable to poisoning and query control attacks, and suggesting improvements in privacy analysis.
Contribution
It presents the first framework for auditing private prediction algorithms using adversaries with varying capabilities and introduces techniques to evaluate privacy leakage in terms of Renyi DP.
Findings
Privacy analysis of private prediction can be improved.
Easier-to-poison algorithms exhibit higher privacy leakage.
Privacy leakage is lower for adversaries without query control.
Abstract
Differential privacy (DP) offers a theoretical upper bound on the potential privacy leakage of analgorithm, while empirical auditing establishes a practical lower bound. Auditing techniques exist forDP training algorithms. However machine learning can also be made private at inference. We propose thefirst framework for auditing private prediction where we instantiate adversaries with varying poisoningand query capabilities. This enables us to study the privacy leakage of four private prediction algorithms:PATE [Papernot et al., 2016], CaPC [Choquette-Choo et al., 2020], PromptPATE [Duan et al., 2023],and Private-kNN [Zhu et al., 2020]. To conduct our audit, we introduce novel techniques to empiricallyevaluate privacy leakage in terms of Renyi DP. Our experiments show that (i) the privacy analysis ofprivate prediction can be improved, (ii) algorithms which are easier to poison lead to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHealthcare Policy and Management · Legal and Constitutional Studies · Medical Malpractice and Liability Issues
