Private Linear Regression with Differential Privacy and PAC Privacy

TL;DR

This paper compares differential privacy and PAC privacy approaches for linear regression, analyzing their performance on real datasets to understand their effectiveness in privacy-preserving statistical modeling.

Contribution

It provides a systematic comparison of differential privacy and PAC privacy in linear regression, highlighting their relative strengths and limitations.

Findings

Differential privacy and PAC privacy yield different trade-offs in model accuracy.

Performance varies significantly across datasets for both privacy methods.

Insights inform better choice of privacy techniques in linear regression applications.

Abstract

Linear regression is a fundamental tool for statistical analysis, which has motivated the development of linear regression methods that satisfy provable privacy guarantees so that the learned model reveals little about any one data point used to construct it. Most existing privacy-preserving linear regression methods rely on the well-established framework of differential privacy, while the newly proposed PAC Privacy has not yet been explored in this context. In this paper, we systematically compare linear regression models trained with differential privacy and PAC privacy across three real-world datasets, observing several key findings that impact the performance of privacy-preserving linear regression.