Differential Privacy with Higher Utility by Exploiting Coordinate-wise Disparity: Laplace Mechanism Can Beat Gaussian in High Dimensions

TL;DR

This paper introduces coordinate-wise disparity-aware noise addition in differential privacy, showing that non-i.i.d. mechanisms like Laplace can outperform Gaussian in high-dimensional settings, enhancing utility across various applications.

Contribution

It formally analyzes and derives conditions for i.n.i.d. Gaussian and Laplace mechanisms, demonstrating their superior utility by exploiting coordinate-wise disparities in privacy leakage.

Findings

i.n.i.d. mechanisms achieve higher utility than i.i.d. ones.

Laplace mechanism can outperform Gaussian in high dimensions.

Improved privacy-utility trade-offs in PCA, coordinate descent, and deep learning.

Abstract

Conventionally, in a differentially private additive noise mechanism, independent and identically distributed (i.i.d.) noise samples are added to each coordinate of the response. In this work, we formally present the addition of noise that is independent but not identically distributed (i.n.i.d.) across the coordinates to achieve tighter privacy-accuracy trade-off by exploiting coordinate-wise disparity in privacy leakage. In particular, we study the i.n.i.d. Gaussian and Laplace mechanisms and obtain the conditions under which these mechanisms guarantee privacy. The optimal choice of parameters that ensure these conditions are derived considering (weighted) mean squared and $\ell_{p}^{p}$-errors as measures of accuracy. Theoretical analyses and numerical simulations demonstrate that the i.n.i.d. mechanisms achieve higher utility for the given privacy requirements compared to their i.i.d. counterparts. One of the interesting observations is that the Laplace mechanism outperforms Gaussian even in high dimensions, as opposed to the popular belief, if the irregularity in coordinate-wise sensitivities is exploited. We also demonstrate how the i.n.i.d. noise can improve the performance in private (a) coordinate descent, (b) principal component analysis, and (c) deep learning with group clipping.