Mitigating Participation Imbalance Bias in Asynchronous Federated Learning

TL;DR

This paper analyzes how asynchronous federated learning amplifies client heterogeneity bias and proposes ACE and ACED methods to mitigate participation imbalance, improving model robustness across diverse data and delay conditions.

Contribution

It provides a theoretical framework linking AFL design choices to heterogeneity amplification and introduces new algorithms to reduce participation bias.

Findings

ACE effectively mitigates participation imbalance.

ACED balances client diversity and update staleness.

Proposed methods outperform baselines in diverse settings.

Abstract

In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client's contribution. As a result, clients perform their local training on different model versions, causing information staleness (delay). In federated environments with non-IID local data distributions, this asynchronous pattern amplifies the adverse effect of client heterogeneity (due to different data distribution, local objectives, etc.), as faster clients contribute more frequent updates, biasing the global model. We term this phenomenon heterogeneity amplification. Our work provides a theoretical analysis that maps AFL design choices to their resulting error sources when heterogeneity amplification occurs. Guided by our analysis, we propose ACE (All-Client Engagement AFL), which mitigates participation imbalance through immediate, non-buffered updates that use the latest information available from all clients. We also introduce a delay-aware variant, ACED, to balance client diversity against update staleness. Experiments on different models for different tasks across diverse heterogeneity and delay settings validate our analysis and demonstrate the robust performance of our approaches.