Private Networked Federated Learning for Nonsmooth Objectives

TL;DR

This paper introduces a privacy-preserving federated learning algorithm for nonsmooth objectives using zCDP, combining ADMM with noise perturbation to ensure confidentiality and convergence guarantees.

Contribution

It develops a novel federated learning method that guarantees privacy with zCDP while effectively handling nonsmooth, non-strongly convex problems using ADMM.

Findings

Achieves a favorable privacy-accuracy trade-off.

Proves convergence to the exact solution.

Demonstrates effectiveness through numerical simulations.

Abstract

This paper develops a networked federated learning algorithm to solve nonsmooth objective functions. To guarantee the confidentiality of the participants with respect to each other and potential eavesdroppers, we use the zero-concentrated differential privacy notion (zCDP). Privacy is achieved by perturbing the outcome of the computation at each client with a variance-decreasing Gaussian noise. ZCDP allows for better accuracy than the conventional $(\epsilon, \delta)$-DP and stronger guarantees than the more recent R\'enyi-DP by assuming adversaries aggregate all the exchanged messages. The proposed algorithm relies on the distributed Alternating Direction Method of Multipliers (ADMM) and uses the approximation of the augmented Lagrangian to handle nonsmooth objective functions. The developed private networked federated learning algorithm has a competitive privacy accuracy trade-off and handles nonsmooth and non-strongly convex problems. We provide complete theoretical proof for the privacy guarantees and the algorithm's convergence to the exact solution. We also prove under additional assumptions that the algorithm converges in $O(1/n)$ ADMM iterations. Finally, we observe the performance of the algorithm in a series of numerical simulations.