Asynchronous Federated Learning on Heterogeneous Devices: A Survey

TL;DR

This survey reviews asynchronous federated learning approaches on heterogeneous devices, highlighting their advantages in efficiency and privacy, and discusses challenges and future research directions in this evolving field.

Contribution

It provides a comprehensive classification and analysis of existing AFL methods tailored for heterogeneous devices, addressing key issues like resource utilization and data heterogeneity.

Findings

AFL approaches improve efficiency on heterogeneous devices.

Data heterogeneity impacts global model accuracy.

Challenges include resource management and privacy concerns.

Abstract

Federated learning (FL) is a kind of distributed machine learning framework, where the global model is generated on the centralized aggregation server based on the parameters of local models, addressing concerns about privacy leakage caused by the collection of local training data. With the growing computational and communication capacities of edge and IoT devices, applying FL on heterogeneous devices to train machine learning models is becoming a prevailing trend. Nonetheless, the synchronous aggregation strategy in the classic FL paradigm, particularly on heterogeneous devices, encounters limitations in resource utilization due to the need to wait for slow devices before aggregation in each training round. Furthermore, the uneven distribution of data across devices (i.e. data heterogeneity) in real-world scenarios adversely impacts the accuracy of the global model. Consequently, many asynchronous FL (AFL) approaches have been introduced across various application contexts to enhance efficiency, performance, privacy, and security. This survey comprehensively analyzes and summarizes existing AFL variations using a novel classification scheme, including device heterogeneity, data heterogeneity, privacy, and security on heterogeneous devices, as well as applications on heterogeneous devices. Finally, this survey reveals rising challenges and presents potentially promising research directions in this under-investigated domain.