Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations

TL;DR

This survey reviews recent advances in heterogeneous federated learning, focusing on challenges, approaches to handle data, model, task, device, and communication heterogeneity, and privacy strategies, highlighting future research directions.

Contribution

It categorizes and analyzes state-of-the-art methods addressing heterogeneity in federated learning across multiple levels and discusses privacy-preserving techniques and open issues.

Findings

Categorization of approaches at data, model, and architecture levels.

Analysis of privacy-preserving strategies in heterogeneous FL.

Identification of open challenges and future research directions.

Abstract

In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and collaboratively train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources. This diversity leads to significant heterogeneity, which increases the complexity of model training. In this paper, we first outline the basic concepts of heterogeneous federated learning and summarize the research challenges in federated learning in terms of five aspects: data, model, task, device, and communication. In addition, we explore how existing state-of-the-art approaches cope with the heterogeneity of federated learning, and categorize and review these approaches at three different levels: data-level, model-level, and architecture-level. Subsequently, the paper extensively discusses privacy-preserving strategies in heterogeneous federated learning environments. Finally, the paper discusses current open issues and directions for future research, aiming to promote the further development of heterogeneous federated learning.