Sliced R\'enyi Pufferfish Privacy: Directional Additive Noise Mechanism and Private Learning with Gradient Clipping

TL;DR

This paper introduces Sliced Renyi Pufferfish Privacy (SRPP), a new privacy framework that improves high-dimensional privacy mechanisms and supports practical iterative learning with better privacy-utility trade-offs.

Contribution

The paper proposes SRPP, a sliced divergence approach for Pufferfish Privacy, along with Gaussian mechanisms and gradient clipping methods for scalable private learning.

Findings

SRPP provides effective privacy guarantees with improved utility.

The proposed mechanisms are scalable and suitable for iterative learning.

Experiments demonstrate favorable privacy-utility trade-offs.

Abstract

We study the design of a privatization mechanism and privacy accounting in the Pufferfish Privacy (PP) family. Specifically, motivated by the curse of dimensionality and lack of practical composition tools for iterative learning in the recent Renyi Pufferfish Privacy (RPP) framework, we propose Sliced Renyi Pufferfish Privacy (SRPP). SRPP preserves PP/RPP semantics (customizable secrets with probability-aware secret-dataset relationships) while replacing high-dimensional Renyi divergence with projection-based quantification via two sliced measures, Average SRPP and Joint SRPP. We develop sliced Wasserstein mechanisms, yielding sound SRPP certificates and closed-form Gaussian noise calibration. For iterative learning systems, we introduce an SRPP-SGD scheme with gradient clipping and new accountants based on History-Uniform Caps (HUC) and a subsampling-aware variant (sa-HUC), enabling decompose-then-compose privatization and additive composition under a common slicing geometry. Experiments on static and iterative privatization show that the proposed framework exhibits favorable privacy-utility trade-offs, as well as practical scalability.