Continuous-Time Analysis of Federated Averaging

TL;DR

This paper extends the convergence analysis of federated averaging (FedAvg) to a continuous-time framework using stochastic differential equations, providing new insights into its convergence, approximation, and generalization properties.

Contribution

It introduces the first continuous-time analysis of FedAvg, broadening understanding of its convergence and generalization in federated learning.

Findings

Convergence guarantees under various loss functions.

Conditions for FedAvg updates to be approximated as normal variables.

Insights into the generalization properties of FedAvg.

Abstract

Federated averaging (FedAvg) is a popular algorithm for horizontal federated learning (FL), where samples are gathered across different clients and are not shared with each other or a central server. Extensive convergence analysis of FedAvg exists for the discrete iteration setting, guaranteeing convergence for a range of loss functions and varying levels of data heterogeneity. We extend this analysis to the continuous-time setting where the global weights evolve according to a multivariate stochastic differential equation (SDE), which is the first time FedAvg has been studied from the continuous-time perspective. We use techniques from stochastic processes to establish convergence guarantees under different loss functions, some of which are more general than existing work in the discrete setting. We also provide conditions for which FedAvg updates to the server weights can be approximated as normal random variables. Finally, we use the continuous-time formulation to reveal generalization properties of FedAvg.