Locally Differentially Private Online Federated Learning With Correlated Noise

TL;DR

This paper presents a novel locally differentially private online federated learning algorithm that uses temporally correlated noise to enhance utility, effectively managing non-IID data and dynamic environments, with theoretical guarantees and empirical validation.

Contribution

It introduces a new LDP algorithm with correlated noise for online federated learning, along with a perturbed iterate analysis and dynamic regret bounds for nonconvex loss functions.

Findings

The proposed method achieves improved utility under LDP constraints.

The algorithm effectively handles streaming non-IID data and environment drift.

Numerical experiments demonstrate the practical effectiveness of the approach.

Abstract

We introduce a locally differentially private (LDP) algorithm for online federated learning that employs temporally correlated noise to improve utility while preserving privacy. To address challenges posed by the correlated noise and local updates with streaming non-IID data, we develop a perturbed iterate analysis that controls the impact of the noise on the utility. Moreover, we demonstrate how the drift errors from local updates can be effectively managed for several classes of nonconvex loss functions. Subject to an $(\epsilon,\delta)$-LDP budget, we establish a dynamic regret bound that quantifies the impact of key parameters and the intensity of changes in the dynamic environment on the learning performance. Numerical experiments confirm the efficacy of the proposed algorithm.