Resource-Aware Asynchronous Online Federated Learning for Nonlinear Regression

TL;DR

This paper introduces a resource-aware asynchronous federated learning method for nonlinear regression that operates reliably in real-world, heterogeneous environments, reducing communication costs while maintaining convergence.

Contribution

It proposes an asynchronous federated learning framework with partial-sharing communication, proven convergence, and significant communication reduction compared to existing methods.

Findings

Achieves convergence comparable to federated SGD in asynchronous settings.

Reduces communication load tenfold through partial-sharing.

Operates effectively with non-IID data and heterogeneous client participation.

Abstract

Many assumptions in the federated learning literature present a best-case scenario that can not be satisfied in most real-world applications. An asynchronous setting reflects the realistic environment in which federated learning methods must be able to operate reliably. Besides varying amounts of non-IID data at participants, the asynchronous setting models heterogeneous client participation due to available computational power and battery constraints and also accounts for delayed communications between clients and the server. To reduce the communication overhead associated with asynchronous online federated learning (ASO-Fed), we use the principles of partial-sharing-based communication. In this manner, we reduce the communication load of the participants and, therefore, render participation in the learning task more accessible. We prove the convergence of the proposed ASO-Fed and provide simulations to analyze its behavior further. The simulations reveal that, in the asynchronous setting, it is possible to achieve the same convergence as the federated stochastic gradient (Online-FedSGD) while reducing the communication tenfold.