On the Differentially Private Nature of Perturbed Gradient Descent

TL;DR

This paper demonstrates that perturbed gradient descent inherently provides differential privacy when used for empirical risk minimization, especially in non-convex optimization scenarios with saddle points.

Contribution

It establishes a theoretical link between gradient perturbation and differential privacy, quantifying privacy guarantees in non-convex optimization.

Findings

Perturbed gradient descent inherently preserves data privacy.

Privacy levels depend on problem dimension and database differences.

The analysis applies to non-convex optimization with saddle points.

Abstract

We consider the problem of empirical risk minimization given a database, using the gradient descent algorithm. We note that the function to be optimized may be non-convex, consisting of saddle points which impede the convergence of the algorithm. A perturbed gradient descent algorithm is typically employed to escape these saddle points. We show that this algorithm, that perturbs the gradient, inherently preserves the privacy of the data. We then employ the differential privacy framework to quantify the privacy hence achieved. We also analyze the change in privacy with varying parameters such as problem dimension and the distance between the databases.