Private Sketches for Linear Regression

TL;DR

This paper introduces a method for creating differentially private sketches of datasets to perform linear regression, enabling privacy-preserving analysis with good approximation guarantees and computational efficiency.

Contribution

It proposes a novel approach of releasing private sketches for linear regression, extending to regularized problems and providing privacy bounds for the regularization parameter.

Findings

Private sketches enable privacy-preserving regression analysis.

The method applies to least squares and least absolute deviations regression.

Privacy guarantees are established for regularized regression solutions.

Abstract

Linear regression is frequently applied in a variety of domains, some of which might contain sensitive information. This necessitates that the application of these methods does not reveal private information. Differentially private (DP) linear regression methods, developed for this purpose, compute private estimates of the solution. These techniques typically involve computing a noisy version of the solution vector. Instead, we propose releasing private sketches of the datasets, which can then be used to compute an approximate solution to the regression problem. This is motivated by the \emph{sketch-and-solve} paradigm, where the regression problem is solved on a smaller sketch of the dataset instead of on the original problem space. The solution obtained on the sketch can also be shown to have good approximation guarantees to the original problem. Various sketching methods have been developed for improving the computational efficiency of linear regression problems under this paradigm. We adopt this paradigm for the purpose of releasing private sketches of the data. We construct differentially private sketches for the problems of least squares regression, as well as least absolute deviations regression. We show that the privacy constraints lead to sketched versions of regularized regression. We compute the bounds on the regularization parameter required for guaranteeing privacy. The availability of these private sketches facilitates the application of commonly available solvers for regression, without the risk of privacy leakage.