Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregation

TL;DR

This paper analyzes the convergence of the FedBuff algorithm in asynchronous federated learning, removing previous boundedness assumptions and considering data heterogeneity, batch size, and delay effects.

Contribution

It extends the theoretical analysis of FedBuff by removing bounded gradient assumptions and incorporating heterogeneity factors.

Findings

Convergence rate established under relaxed assumptions

Handles data heterogeneity and asynchronous delays

Provides theoretical guarantees for FedBuff performance

Abstract

Synchronous updates may compromise the efficiency of cross-device federated learning once the number of active clients increases. The \textit{FedBuff} algorithm (Nguyen et al., 2022) alleviates this problem by allowing asynchronous updates (staleness), which enhances the scalability of training while preserving privacy via secure aggregation. We revisit the \textit{FedBuff} algorithm for asynchronous federated learning and extend the existing analysis by removing the boundedness assumptions from the gradient norm. This paper presents a theoretical analysis of the convergence rate of this algorithm when heterogeneity in data, batch size, and delay are considered.