FedECADO: A Dynamical System Model of Federated Learning

TL;DR

FedECADO introduces a novel dynamical system-inspired algorithm for federated learning that effectively manages data heterogeneity and computational variability, leading to improved accuracy over existing methods.

Contribution

It presents FedECADO, a new federated learning algorithm using dynamical systems theory to handle non-IID data and heterogeneous computing environments.

Findings

FedECADO outperforms FedProx and FedNova in accuracy across various heterogeneous scenarios.

The aggregate sensitivity model effectively captures data distribution differences.

Adaptive multi-rate integration improves synchronization of client updates.

Abstract

Federated learning harnesses the power of distributed optimization to train a unified machine learning model across separate clients. However, heterogeneous data distributions and computational workloads can lead to inconsistent updates and limit model performance. This work tackles these challenges by proposing FedECADO, a new algorithm inspired by a dynamical system representation of the federated learning process. FedECADO addresses non-IID data distribution through an aggregate sensitivity model that reflects the amount of data processed by each client. To tackle heterogeneous computing, we design a multi-rate integration method with adaptive step-size selections that synchronizes active client updates in continuous time. Compared to prominent techniques, including FedProx and FedNova, FedECADO achieves higher classification accuracies in numerous heterogeneous scenarios.