On the Stability Analysis of Open Federated Learning Systems

TL;DR

This paper introduces a new stability metric for open federated learning systems where clients can join or leave, and provides theoretical analysis of the stability radius for local SGD and Adam algorithms.

Contribution

It proposes a novel stability measure for open FL systems and derives theoretical bounds for local SGD and Adam under convexity and smoothness assumptions.

Findings

Stability radius depends on function condition number and gradient variance.

Theoretical bounds are validated through simulations on synthetic and real data.

Open FL systems can maintain a bounded model stability despite client variability.

Abstract

We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process. Given the variability of the number of present clients, convergence to a fixed model cannot be guaranteed in open systems. Instead, we resort to a new performance metric that we term the stability of open FL systems, which quantifies the magnitude of the learned model in open systems. Under the assumption that local clients' functions are strongly convex and smooth, we theoretically quantify the radius of stability for two FL algorithms, namely local SGD and local Adam. We observe that this radius relies on several key parameters, including the function condition number as well as the variance of the stochastic gradient. Our theoretical results are further verified by numerical simulations on both synthetic and real-world benchmark data-sets.