Towards Heterogeneous Clients with Elastic Federated Learning

TL;DR

This paper introduces Elastic Federated Learning (EFL), an algorithm designed to address heterogeneity and non-IID data in federated learning, ensuring unbiased training, reduced communication costs, and convergence guarantees.

Contribution

The paper proposes EFL, a novel federated learning algorithm that handles heterogeneity and incomplete updates while providing theoretical convergence guarantees.

Findings

EFL achieves robust performance on non-IID data.

EFL reduces communication overhead effectively.

EFL demonstrates convergence guarantees under low participation rates.

Abstract

Federated learning involves training machine learning models over devices or data silos, such as edge processors or data warehouses, while keeping the data local. Training in heterogeneous and potentially massive networks introduces bias into the system, which is originated from the non-IID data and the low participation rate in reality. In this paper, we propose Elastic Federated Learning (EFL), an unbiased algorithm to tackle the heterogeneity in the system, which makes the most informative parameters less volatile during training, and utilizes the incomplete local updates. It is an efficient and effective algorithm that compresses both upstream and downstream communications. Theoretically, the algorithm has convergence guarantee when training on the non-IID data at the low participation rate. Empirical experiments corroborate the competitive performance of EFL framework on the robustness and the efficiency.