Dynamic Privacy Allocation for Locally Differentially Private Federated Learning with Composite Objectives

TL;DR

This paper introduces a novel federated learning algorithm that dynamically allocates privacy-preserving noise to optimize utility while maintaining strong privacy guarantees, applicable to convex problems with nonsmooth objectives.

Contribution

It presents a new privacy allocation method that adapts noise variance over iterations, eliminating the need for iteration tuning in private federated learning.

Findings

Outperforms existing methods in numerical experiments

Achieves privacy and utility up to a neighborhood of the optimal solution

Supports strongly convex, possibly nonsmooth problems

Abstract

This paper proposes a locally differentially private federated learning algorithm for strongly convex but possibly nonsmooth problems that protects the gradients of each worker against an honest but curious server. The proposed algorithm adds artificial noise to the shared information to ensure privacy and dynamically allocates the time-varying noise variance to minimize an upper bound of the optimization error subject to a predefined privacy budget constraint. This allows for an arbitrarily large but finite number of iterations to achieve both privacy protection and utility up to a neighborhood of the optimal solution, removing the need for tuning the number of iterations. Numerical results show the superiority of the proposed algorithm over state-of-the-art methods.