Residual-PAC Privacy: Automatic Privacy Control Beyond the Gaussian Barrier

TL;DR

This paper introduces Residual-PAC Privacy, a novel framework that improves privacy guarantees beyond Gaussian assumptions by using f-divergence measures and game-theoretic optimization, leading to better privacy-utility tradeoffs.

Contribution

It proposes Residual-PAC Privacy with a practical Stackelberg mechanism, overcoming Gaussian limitations and enhancing privacy budget efficiency in complex data distributions.

Findings

SR-PAC outperforms PAC and differential privacy in experiments

Efficient privacy budget utilization for arbitrary data distributions

Natural composition of multiple privacy mechanisms

Abstract

The Probably Approximately Correct (PAC) Privacy framework [46] provides a powerful instance-based methodology to preserve privacy in complex data-driven systems. Existing PAC Privacy algorithms (we call them Auto-PAC) rely on a Gaussian mutual information upper bound. However, we show that the upper bound obtained by these algorithms is tight if and only if the perturbed mechanism output is jointly Gaussian with independent Gaussian noise. We propose two approaches for addressing this issue. First, we introduce two tractable post-processing methods for Auto-PAC, based on Donsker-Varadhan representation and sliced Wasserstein distances. However, the result still leaves wasted privacy budget. To address this issue more fundamentally, we introduce Residual-PAC (R-PAC) Privacy, an f-divergence-based measure to quantify privacy that remains after adversarial inference. To implement R-PAC Privacy in practice, we propose a Stackelberg Residual-PAC (SR-PAC) privatization mechanism, a game-theoretic framework that selects optimal noise distributions through convex bilevel optimization. Our approach achieves efficient privacy budget utilization for arbitrary data distributions and naturally composes when multiple mechanisms access the dataset. Our experiments demonstrate that SR-PAC consistently obtains a better privacy-utility tradeoff than both PAC and differential privacy baselines.