The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing
Meenatchi Sundaram Muthu Selva Annamalai, Borja Balle, Jamie Hayes, Georgios Kaissis, Emiliano De Cristofaro

TL;DR
This paper reviews and systematizes differential privacy auditing techniques, focusing on efficiency, end-to-end process, and tightness, and identifies open challenges and future research directions.
Contribution
It introduces a comprehensive framework for evaluating DP audits and analyzes current methods to highlight limitations and open problems.
Findings
Highlights overlooked details in current DP auditing techniques.
Identifies key limiting factors to achieving efficiency, end-to-end-ness, and tightness.
Provides a systematic methodology for assessing progress in DP auditing.
Abstract
In this paper, we systematize research on auditing Differential Privacy (DP) techniques, aiming to identify key insights and open challenges. First, we introduce a comprehensive framework for reviewing work in the field and establish three cross-contextual desiderata that DP audits should target -- namely, efficiency, end-to-end-ness, and tightness. Then, we systematize the modes of operation of state-of-the-art DP auditing techniques, including threat models, attacks, and evaluation functions. This allows us to highlight key details overlooked by prior work, analyze the limiting factors to achieving the three desiderata, and identify open research problems. Overall, our work provides a reusable and systematic methodology geared to assess progress in the field and identify friction points and future directions for our community to focus on.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Adversarial Robustness in Machine Learning · Internet Traffic Analysis and Secure E-voting
MethodsFocus
