Perturb-and-Project: Differentially Private Similarities and Marginals

TL;DR

This paper introduces new algorithms within the perturb-and-project framework to privately compute similarities and marginals, extending capabilities to sparse datasets and providing theoretical insights into their practical effectiveness.

Contribution

It presents novel, efficient algorithms for differentially private similarity and marginal computations, including for sparse datasets, with theoretical analysis of their success.

Findings

Efficient algorithms for private pairwise cosine similarities.

Algorithms for $k$-way marginal queries over $n$ features.

Stronger guarantees for $t$-sparse datasets when $t \

Abstract

We revisit the input perturbations framework for differential privacy where noise is added to the input $A\in \mathcal{S}$ and the result is then projected back to the space of admissible datasets $\mathcal{S}$. Through this framework, we first design novel efficient algorithms to privately release pair-wise cosine similarities. Second, we derive a novel algorithm to compute $k$-way marginal queries over $n$ features. Prior work could achieve comparable guarantees only for $k$ even. Furthermore, we extend our results to $t$-sparse datasets, where our efficient algorithms yields novel, stronger guarantees whenever $t\le n^{5/6}/\log n\,.$ Finally, we provide a theoretical perspective on why \textit{fast} input perturbation algorithms works well in practice. The key technical ingredients behind our results are tight sum-of-squares certificates upper bounding the Gaussian complexity of sets of solutions.