Refined Differentially Private Linear Regression via Extension of a Free Lunch Result

TL;DR

This paper extends a recent 'free' privacy-preserving technique for linear regression by developing multidimensional transformations that improve the estimation of sufficient statistics under differential privacy.

Contribution

It introduces new multidimensional simplex transformations for bounded variables, refining private estimates for linear regression and potentially applicable to polynomial regression.

Findings

Transformations improve accuracy of private linear regression estimates.

Analytical and numerical results demonstrate the method's superiority.

Transformations are adaptable for differentially private polynomial regression.

Abstract

As data-privacy regulations tighten and statistical models are increasingly deployed on sensitive human-sourced data, privacy-preserving linear regression has become a critical necessity. For the add-remove DP model, Kulesza et al. (2024) and Fitzsimons et al. (2024) have independently shown that the size of the dataset -- an important statistic for linear regression -- can be privately estimated for "free", via a simplex transformation of bounded variables and private sum queries on the transformed variables. In this work, we extend this free lunch result via carefully crafted multidimensional simplex transformations to variables and functions that are bounded in the interval [0,1]. We show that these transformations can be applied to refine the estimates of sufficient statistics needed for private simple linear regression based on ordinary least squares. We provide both analytical and numerical results to demonstrate the superiority of our approach. Our proposed transformations have general applicability and can be readily adapted for differentially private polynomial regression.