From Robustness to Privacy and Back

TL;DR

This paper establishes a novel black-box transformation converting adversarially robust algorithms into differentially private ones, achieving optimal error rates in low-dimensional estimation and extending to high-dimensional tasks like Gaussian linear regression and PCA.

Contribution

It introduces the first black-box method for transforming robust algorithms into pure differentially private algorithms, matching optimal error rates in low-dimensional settings.

Findings

Optimal private estimators match robust estimator error rates in low dimensions.

New private estimators for Gaussian linear regression and PCA.

Extension to approximate privacy with error independent of output range.

Abstract

We study the relationship between two desiderata of algorithms in statistical inference and machine learning: differential privacy and robustness to adversarial data corruptions. Their conceptual similarity was first observed by Dwork and Lei (STOC 2009), who observed that private algorithms satisfy robustness, and gave a general method for converting robust algorithms to private ones. However, all general methods for transforming robust algorithms into private ones lead to suboptimal error rates. Our work gives the first black-box transformation that converts any adversarially robust algorithm into one that satisfies pure differential privacy. Moreover, we show that for any low-dimensional estimation task, applying our transformation to an optimal robust estimator results in an optimal private estimator. Thus, we conclude that for any low-dimensional task, the optimal error rate for $\varepsilon$-differentially private estimators is essentially the same as the optimal error rate for estimators that are robust to adversarially corrupting $1/\varepsilon$ training samples. We apply our transformation to obtain new optimal private estimators for several high-dimensional tasks, including Gaussian (sparse) linear regression and PCA. Finally, we present an extension of our transformation that leads to approximate differentially private algorithms whose error does not depend on the range of the output space, which is impossible under pure differential privacy.