# Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

## TL;DR

This paper investigates the privacy-utility trade-off in linear regression when data is transformed via random projections or additive noise, using a strong differential privacy notion and analyzing the impact on model accuracy.

## Contribution

It introduces a novel analysis of privacy-utility trade-offs for linear regression under MI-DP and compares the effectiveness of random projections versus additive noise.

## Key findings

- Random projections generally improve the privacy-utility trade-off.
- Projection before noise addition yields better utility than noise alone.
- Connections to wireless communication enhance the analysis approach.

## Abstract

Data privacy is an important concern in machine learning, and is fundamentally at odds with the task of training useful learning models, which typically require the acquisition of large amounts of private user data. One possible way of fulfilling the machine learning task while preserving user privacy is to train the model on a transformed, noisy version of the data, which does not reveal the data itself directly to the training procedure. In this work, we analyze the privacy-utility trade-off of two such schemes for the problem of linear regression: additive noise, and random projections. In contrast to previous work, we consider a recently proposed notion of differential privacy that is based on conditional mutual information (MI-DP), which is stronger than the conventional $(\epsilon, \delta)$-differential privacy, and use relative objective error as the utility metric. We find that projecting the data to a lower-dimensional subspace before adding noise attains a better trade-off in general. We also make a connection between privacy problem and (non-coherent) SIMO, which has been extensively studied in wireless communication, and use tools from there for the analysis. We present numerical results demonstrating the performance of the schemes.