Differentially Private Algorithms for Empirical Machine Learning

TL;DR

This paper introduces practical differentially private algorithms for training classifiers, feature selection, and ROC curve construction, improving accuracy and enabling private evaluation on real-world datasets.

Contribution

It presents novel private algorithms for feature selection and ROC curve construction that enhance the practicality of differentially private machine learning workflows.

Findings

Significant accuracy improvements on three real-world datasets.

First private algorithms for ROC curve construction.

Effective feature selection under differential privacy.

Abstract

An important use of private data is to build machine learning classifiers. While there is a burgeoning literature on differentially private classification algorithms, we find that they are not practical in real applications due to two reasons. First, existing differentially private classifiers provide poor accuracy on real world datasets. Second, there is no known differentially private algorithm for empirically evaluating the private classifier on a private test dataset. In this paper, we develop differentially private algorithms that mirror real world empirical machine learning workflows. We consider the private classifier training algorithm as a blackbox. We present private algorithms for selecting features that are input to the classifier. Though adding a preprocessing step takes away some of the privacy budget from the actual classification process (thus potentially making it noisier and less accurate), we show that our novel preprocessing techniques significantly increase classifier accuracy on three real-world datasets. We also present the first private algorithms for empirically constructing receiver operating characteristic (ROC) curves on a private test set.