Privacy Induces Robustness: Information-Computation Gaps and Sparse Mean Estimation

TL;DR

This paper reveals that differentially-private algorithms are inherently robust against adversarial data corruption, and explores the fundamental tradeoffs and gaps between privacy, computational efficiency, and statistical accuracy across various problems.

Contribution

It establishes a connection between privacy and robustness, demonstrates a privacy-induced information-computation gap for sparse mean estimation, and introduces new mechanisms and evidence for such gaps in multiple learning tasks.

Findings

Private mechanisms with high success probability are robust to adversarial corruption.

A fundamental tradeoff exists between efficiency, privacy leakage, and success probability under certain conjectures.

New exponential-time mechanisms show fewer samples are needed than efficient algorithms for sparse mean estimation.

Abstract

We establish a simple connection between robust and differentially-private algorithms: private mechanisms which perform well with very high probability are automatically robust in the sense that they retain accuracy even if a constant fraction of the samples they receive are adversarially corrupted. Since optimal mechanisms typically achieve these high success probabilities, our results imply that optimal private mechanisms for many basic statistics problems are robust. We investigate the consequences of this observation for both algorithms and computational complexity across different statistical problems. Assuming the Brennan-Bresler secret-leakage planted clique conjecture, we demonstrate a fundamental tradeoff between computational efficiency, privacy leakage, and success probability for sparse mean estimation. Private algorithms which match this tradeoff are not yet known -- we achieve that (up to polylogarithmic factors) in a polynomially-large range of parameters via the Sum-of-Squares method. To establish an information-computation gap for private sparse mean estimation, we also design new (exponential-time) mechanisms using fewer samples than efficient algorithms must use. Finally, we give evidence for privacy-induced information-computation gaps for several other statistics and learning problems, including PAC learning parity functions and estimation of the mean of a multivariate Gaussian.