Sparse Private LASSO Logistic Regression

TL;DR

This paper introduces a differentially private LASSO logistic regression method that preserves sparsity, enabling effective feature selection while maintaining privacy, demonstrated through experiments on synthetic and real datasets.

Contribution

The paper proposes a novel differentially private approach that retains the sparsity of LASSO logistic regression, unlike previous dense solutions, by using a non-private model to guide the privacy-preserving selection.

Findings

Maintains sparsity in private logistic regression models.

Effective feature selection under differential privacy.

Validated on synthetic and real-world datasets.

Abstract

LASSO regularized logistic regression is particularly useful for its built-in feature selection, allowing coefficients to be removed from deployment and producing sparse solutions. Differentially private versions of LASSO logistic regression have been developed, but generally produce dense solutions, reducing the intrinsic utility of the LASSO penalty. In this paper, we present a differentially private method for sparse logistic regression that maintains hard zeros. Our key insight is to first train a non-private LASSO logistic regression model to determine an appropriate privatized number of non-zero coefficients to use in final model selection. To demonstrate our method's performance, we run experiments on synthetic and real-world datasets.