Differentially Private ADMM for Distributed Medical Machine Learning

TL;DR

This paper introduces a differentially private ADMM algorithm for distributed medical machine learning that maintains convergence rates while ensuring data privacy through Gaussian noise, outperforming existing methods.

Contribution

It proposes a novel P-ADMM algorithm that achieves dynamic zCDP guarantees with linear decay noise, matching non-private convergence rates.

Findings

P-ADMM maintains $ ext{O}(1/K)$ convergence rate.

It achieves linear convergence for convex problems.

Empirical results show superior performance over existing private ADMM algorithms.

Abstract

Due to massive amounts of data distributed across multiple locations, distributed machine learning has attracted a lot of research interests. Alternating Direction Method of Multipliers (ADMM) is a powerful method of designing distributed machine learning algorithm, whereby each agent computes over local datasets and exchanges computation results with its neighbor agents in an iterative procedure. There exists significant privacy leakage during this iterative process if the local data is sensitive. In this paper, we propose a differentially private ADMM algorithm (P-ADMM) to provide dynamic zero-concentrated differential privacy (dynamic zCDP), by inserting Gaussian noise with linearly decaying variance. We prove that P-ADMM has the same convergence rate compared to the non-private counterpart, i.e., $\mathcal{O}(1/K)$ with $K$ being the number of iterations and linear convergence for general convex and strongly convex problems while providing differentially private guarantee. Moreover, through our experiments performed on real-world datasets, we empirically show that P-ADMM has the best-known performance among the existing differentially private ADMM based algorithms.