Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners

TL;DR

This paper enhances the objective perturbation method for differentially private linear models by providing tighter privacy guarantees and computational improvements, making it more practical and competitive with DP-SGD.

Contribution

It introduces a revamped objective perturbation mechanism with improved privacy analysis and computational tools for better performance on linear models.

Findings

Achieves tighter privacy bounds for objective perturbation.

Reduces computational complexity for linear models.

Demonstrates competitive performance with DP-SGD.

Abstract

In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems.