Differentially Private Linear Regression with Linked Data

TL;DR

This paper introduces two differentially private algorithms for linear regression on linked data, addressing the additional uncertainty from record linkage and analyzing the privacy-accuracy tradeoff with finite-sample error bounds.

Contribution

It presents novel privacy-preserving algorithms specifically designed for linked data in linear regression, incorporating linkage uncertainty into the privacy framework.

Findings

Finite-sample error bounds for estimators

Analysis of linkage error impact on accuracy

Demonstrated algorithms via simulations and synthetic data

Abstract

There has been increasing demand for establishing privacy-preserving methodologies for modern statistics and machine learning. Differential privacy, a mathematical notion from computer science, is a rising tool offering robust privacy guarantees. Recent work focuses primarily on developing differentially private versions of individual statistical and machine learning tasks, with nontrivial upstream pre-processing typically not incorporated. An important example is when record linkage is done prior to downstream modeling. Record linkage refers to the statistical task of linking two or more data sets of the same group of entities without a unique identifier. This probabilistic procedure brings additional uncertainty to the subsequent task. In this paper, we present two differentially private algorithms for linear regression with linked data. In particular, we propose a noisy gradient method and a sufficient statistics perturbation approach for the estimation of regression coefficients. We investigate the privacy-accuracy tradeoff by providing finite-sample error bounds for the estimators, which allows us to understand the relative contributions of linkage error, estimation error, and the cost of privacy. The variances of the estimators are also discussed. We demonstrate the performance of the proposed algorithms through simulations and an application to synthetic data.