Differentially Private Iterative Screening Rules for Linear Regression

TL;DR

This paper introduces the first differentially private screening rule for linear regression, balancing privacy and feature elimination to improve sparse model training.

Contribution

It develops a novel private screening rule for linear regression and proposes a weakened implementation to reduce overscreening and enhance performance.

Findings

Private screening rule effectively eliminates irrelevant features.

Weakened screening reduces overscreening and improves model accuracy.

First application of differential privacy to screening in linear models.

Abstract

Linear $L_1$-regularized models have remained one of the simplest and most effective tools in data science. Over the past decade, screening rules have risen in popularity as a way to eliminate features when producing the sparse regression weights of $L_1$ models. However, despite the increasing need of privacy-preserving models for data analysis, to the best of our knowledge, no differentially private screening rule exists. In this paper, we develop the first private screening rule for linear regression. We initially find that this screening rule is too strong: it screens too many coefficients as a result of the private screening step. However, a weakened implementation of private screening reduces overscreening and improves performance.