FedAST: Federated Asynchronous Simultaneous Training

TL;DR

FedAST is a novel asynchronous federated learning algorithm that enables efficient simultaneous training of multiple models across heterogeneous clients, reducing training time and overcoming synchronization bottlenecks.

Contribution

The paper introduces FedAST, an asynchronous federated training method that handles multiple models simultaneously with theoretical guarantees and improved efficiency.

Findings

Achieves up to 46% reduction in training time.

Outperforms existing methods in real-world datasets.

Effectively manages heterogeneous client resources.

Abstract

Federated Learning (FL) enables edge devices or clients to collaboratively train machine learning (ML) models without sharing their private data. Much of the existing work in FL focuses on efficiently learning a model for a single task. In this paper, we study simultaneous training of multiple FL models using a common set of clients. The few existing simultaneous training methods employ synchronous aggregation of client updates, which can cause significant delays because large models and/or slow clients can bottleneck the aggregation. On the other hand, a naive asynchronous aggregation is adversely affected by stale client updates. We propose FedAST, a buffered asynchronous federated simultaneous training algorithm that overcomes bottlenecks from slow models and adaptively allocates client resources across heterogeneous tasks. We provide theoretical convergence guarantees for FedAST for smooth non-convex objective functions. Extensive experiments over multiple real-world datasets demonstrate that our proposed method outperforms existing simultaneous FL approaches, achieving up to 46.0% reduction in time to train multiple tasks to completion.