A Resource-Adaptive Approach for Federated Learning under Resource-Constrained Environments

TL;DR

This paper introduces Fed-RAA, a novel resource-adaptive asynchronous federated learning algorithm that allocates model fragments based on client resources, ensuring convergence and improving efficiency in resource-constrained environments.

Contribution

It proposes the first resource-adaptive asynchronous fragment-based federated learning method with theoretical convergence guarantees.

Findings

Demonstrates improved performance on MNIST, CIFAR-10, and CIFAR-100 datasets.

Shows the effectiveness of online greedy fragment allocation for fairness.

Validates convergence and efficiency through theoretical analysis and experiments.

Abstract

The paper studies a fundamental federated learning (FL) problem involving multiple clients with heterogeneous constrained resources. Compared with the numerous training parameters, the computing and communication resources of clients are insufficient for fast local training and real-time knowledge sharing. Besides, training on clients with heterogeneous resources may result in the straggler problem. To address these issues, we propose Fed-RAA: a Resource-Adaptive Asynchronous Federated learning algorithm. Different from vanilla FL methods, where all parameters are trained by each participating client regardless of resource diversity, Fed-RAA adaptively allocates fragments of the global model to clients based on their computing and communication capabilities. Each client then individually trains its assigned model fragment and asynchronously uploads the updated result. Theoretical analysis confirms the convergence of our approach. Additionally, we design an online greedy-based algorithm for fragment allocation in Fed-RAA, achieving fairness comparable to an offline strategy. We present numerical results on MNIST, CIFAR-10, and CIFAR-100, along with necessary comparisons and ablation studies, demonstrating the advantages of our work. To the best of our knowledge, this paper represents the first resource-adaptive asynchronous method for fragment-based FL with guaranteed theoretical convergence.