Keep It Simple: Fault Tolerance Evaluation of Federated Learning with Unreliable Clients

TL;DR

This paper investigates the fault tolerance of federated learning with unreliable clients, revealing that simple algorithms can perform well despite device dropouts and misconfigurations in real-world scenarios.

Contribution

It provides an empirical analysis of federated learning's robustness against unreliable clients using real-world classification tasks, highlighting the surprising resilience of simple algorithms.

Findings

Simple FL algorithms perform well with unreliable clients

Unreliable devices have limited impact on FL model accuracy

Real-world experiments validate FL fault tolerance

Abstract

Federated learning (FL), as an emerging artificial intelligence (AI) approach, enables decentralized model training across multiple devices without exposing their local training data. FL has been increasingly gaining popularity in both academia and industry. While research works have been proposed to improve the fault tolerance of FL, the real impact of unreliable devices (e.g., dropping out, misconfiguration, poor data quality) in real-world applications is not fully investigated. We carefully chose two representative, real-world classification problems with a limited numbers of clients to better analyze FL fault tolerance. Contrary to the intuition, simple FL algorithms can perform surprisingly well in the presence of unreliable clients.