Federated Learning Aggregation: New Robust Algorithms with Guarantees

TL;DR

This paper analyzes federated learning aggregation algorithms, providing convergence guarantees and introducing new robust methods that adapt to client contributions based on their losses, improving performance in diverse data settings.

Contribution

The paper offers a comprehensive convergence analysis and proposes novel aggregation algorithms that adjust client contributions according to their losses, enhancing robustness.

Findings

New aggregation algorithms outperform FedAvg in various settings.

Algorithms adapt to client loss values for improved robustness.

Performance validated on classification tasks with IID and Non-IID data.

Abstract

Federated Learning has been recently proposed for distributed model training at the edge. The principle of this approach is to aggregate models learned on distributed clients to obtain a new more general "average" model (FedAvg). The resulting model is then redistributed to clients for further training. To date, the most popular federated learning algorithm uses coordinate-wise averaging of the model parameters for aggregation. In this paper, we carry out a complete general mathematical convergence analysis to evaluate aggregation strategies in a federated learning framework. From this, we derive novel aggregation algorithms which are able to modify their model architecture by differentiating client contributions according to the value of their losses. Moreover, we go beyond the assumptions introduced in theory, by evaluating the performance of these strategies and by comparing them with the one of FedAvg in classification tasks in both the IID and the Non-IID framework without additional hypothesis.