A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection

TL;DR

This survey comprehensively reviews federated learning systems, analyzing their components, categorization, and challenges, to guide future research in privacy-preserving collaborative machine learning.

Contribution

It introduces a systematic categorization of federated learning systems based on six aspects and provides a detailed analysis of their design factors and future opportunities.

Findings

Categorization of federated learning systems into six aspects.

Analysis of system components and design factors.

Identification of future research directions.

Abstract

Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing systems and infrastructures to ease the development of various federated learning algorithms. Similar to deep learning systems such as PyTorch and TensorFlow that boost the development of deep learning, federated learning systems (FLSs) are equivalently important, and face challenges from various aspects such as effectiveness, efficiency, and privacy. In this survey, we conduct a comprehensive review on federated learning systems. To achieve smooth flow and guide future research, we introduce the definition of federated learning systems and analyze the system components. Moreover, we provide a thorough categorization for federated learning systems according to six different aspects, including data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation and motivation of federation. The categorization can help the design of federated learning systems as shown in our case studies. By systematically summarizing the existing federated learning systems, we present the design factors, case studies, and future research opportunities.

Tables

Table 1. Table 1: Comparison among existing published studies. LM denotes Linear Models. DM denotes Decision Trees. NN denotes Neural Networks. CM denotes Cryptographic Methods. DP denotes Differential Privacy.

FL Studies

main area

data partitioning

model implementation

privacy mechanism

communication architecture

remark

FedAvg [129]

Effective Algorithms

horizontal

NN

centralized

SGD-based

FedSVRG [94]

LM

FedProx [108]

LM, NN

SCAFFOLD [90]

LM, NN

FedNova [190]

NN

Per-FedAvg [52]

NN

pFedMe [46]

LM, NN

IAPGD, AL2SGD+ [69]

LM

IFCA [61]

LM, NN

Agnostic FL [134]

LM, NN

FedRobust [155]

NN

FedDF [114]

NN

\cdashline2-2                                                          FedBCD [120]

vertical

NN

\cdashline4-6 PNFM [213]

horizontal

NN-specialized

FedMA [189]

SplitNN [189]

vertical

\cdashline2-2                                                          Tree-based FL [217]

horizontal

DT

DP

decentralized

DT-specialized

SimFL [104]

hashing

\cdashline2-4 FedXGB [122]

CM

centralized

FedForest [121]

SecureBoost [38]

vertical

\cdashline2-2\cdashline5-6                                                          Ridge Regression FL [141]

horizontal

LM

LM-specialized

PPRR [36]

Linear Regression FL [162]

vertical

Logistic Regression FL [72]

horizontal

\cdashline2-4\cdashline6-6 Federated MTL [169]

multi-task learning

\cdashline2-3\cdashline5-6 Federated Meta-Learning [33]

NN

meta-learning

Personalized FedAvg [81]

\cdashline2-6 LFRL [115]

reinforcement learning

FBO [44]

LM

Bayesian optimization

\cdashline3-3\cdashline5-6 Structure Updates [95]

Practicality Enhancement

NN

efficiency improvement

Multi-Objective FL [226]

On-Device ML [79]

Sparse Ternary Compression [164]

\cdashline2-5 DPASGD [128]

decentralized

\cdashline2-4 Client-Level DP FL [60]

DP

centralized

privacy guarantees

FL-LSTM [130]

Local DP FL [20]

LM, NN

Secure Aggregation FL [23]

NN

CM

Hybrid FL [181]

LM, DT, NN

CM, DP

\cdashline2-3\cdashline6-6 Backdoor FL [16, 174, 188]

NN

robustness and attacks

Adversarial Lens [19]

Distributed Backdoor [203]

Image Reconstruction [58]

\cdashline2-3 RSA [100]

LM

Model Poison [53]

LM, NN

\cdashline5-6\cdashline2-3 $q$-FedAvg [110]

LM, NN

fairness

BlockFL [93]

LM

incentives

Reputation FL [87]

\cdashline3-3\cdashline5-6 FedCS [143]

Applications

NN

edge computing

DRL-MEC [194]

Resource-Constrained MEC [192]

LM, NN

FedGKT [73]

NN

\cdashline2-3\cdashline5-6 FedCF [14]

LM

collaborative filter

FedMF [29]

matrix factorization

\cdashline2-3\cdashline6-6 FedRecSys [177]

LM, NN

CM

recommender system

FL Keyboard [71]

NN

natural language processing

Fraud detection [222]

NN

credit card transaction

\cdashline5-5 FedML [74]

Benchmarks

horizontal &vertical

LM, NN

centralized &decentralized

general purpose benchmarks

FedEval [30]

horizontal

NN

centralized

OARF [77]

NN

CM,DP

centralized

Edge AIBench [70]

\cdashline2-3\cdashline5-5 PerfEval [142]

NN

centralized

targeted benchmarks

FedReID [227]

semi-supervised benchmark [216]

non-IID benchmark [117]

\cdashline2-5 LEAF [25]

centralized

datasets

Street Dataset [124]

Table 2. Table 2: The comparison among some existing FLSs. The notations used in this table are the same as Table  4.2 . The cell is left empty if the system does not support the corresponding feature. There is no release version for FedML.

Supported features		FATE 1.5.0	TFF 0.17.0	PySyft 0.3.0	PaddleFL 1.1.0	FedML
Operation systems	Mac	✓	✓	✓	✓	✓
	Linux	✓	✓	✓	✓	✓
	Windows			✓	✓	✓
	iOS					✓
	Android					✓
Data partitioning	horizontal	✓	✓	✓	✓	✓
	vertical	✓			✓	✓
Models	NN	✓	✓	✓	✓	✓
	DT	✓
	LM	✓	✓	✓	✓	✓
Privacy Mechanisms	DP		✓	✓	✓	✓
	CM	✓		✓	✓
Communication	simulated	✓	✓	✓	✓	✓
	distributed	✓		✓	✓	✓
Hardwares	CPUs	✓	✓	✓	✓	✓
	GPUs		✓	✓		✓

Table 3. Table 3: Requirements of the real-world federated systems

System Aspect	Mobile Service	Healthcare	Financial
Data Partitioning	Horizontal Partitioning	Hybrid Partitioning	Vertical Partitioning
Machine Learning Model	No specific Models	No specific Models	No specific Models
Scale of Federations	Cross-device	Cross-silo	Cross-silo
Communication Architecture	Centralized	Distributed	Distributed
Privacy Mechanism	DP	DP/SMC	DP/SMC
Motivation of Federation	Incentive Motivated	Policy Motivated	Interest Motivated

Full text

1 Introduction

Many machine learning algorithms are data hungry, and in reality, data are dispersed over different organizations under the protection of privacy restrictions. Due to these factors, federated learning (FL) [129, 207, 85] has become a hot research topic in machine learning. For example, data of different hospitals are isolated and become “data islands”. Since each data island has limitations in size and approximating real distributions, a single hospital may not be able to train a high-quality model that has a good predictive accuracy for a specific task. Ideally, hospitals can benefit more if they can collaboratively train a machine learning model on the union of their data. However, the data cannot simply be shared among the hospitals due to various policies and regulations. Such phenomena on “data islands” are commonly seen in many areas such as finance, government, and supply chains. Policies such as General Data Protection Regulation (GDPR) [10] stipulate rules on data sharing among different organizations. Thus, it is challenging to develop a federated learning system which has a good predictive accuracy while obeying policies and regulations to protect privacy.

Many efforts have recently been devoted to implementing federated learning algorithms to support effective machine learning models. Specifically, researchers try to support more machine learning models with different privacy-preserving approaches, including deep neural networks (NNs) [119, 213, 24, 158, 129], gradient boosted decision trees (GBDTs) [217, 38, 104], logistics regression [141, 36] and support vector machines (SVMs) [169]. For instance, Nikolaenko et al. [141] and Chen et al. [36] propose approaches to conduct FL based on linear regression. Since GBDTs have become very successful in recent years [34, 200], the corresponding Federated Learning Systems (FLSs) have also been proposed by Zhao et al. [217], Cheng et al. [38], Li et al. [104]. Moreover, there are many FLSs supporting the training of NNs. Google proposes a scalable production system which enables tens of millions of devices to train a deep neural network [24].

As there are common methods and building blocks (e.g., privacy mechanisms such as differential privacy) for building FL algorithms, it makes sense to develop systems and infrastructures to ease the development of various FL algorithms. Systems and infrastructures allow algorithm developers to reuse the common building blocks, and avoid building algorithms every time from scratch. Similar to deep learning systems such as PyTorch [148, 149] and TensorFlow [7] that boost the development of deep learning algorithms, FLSs are equivalently important for the success of FL. However, building a successful FLS is challenging, which needs to consider multiple aspects such as effectiveness, efficiency, privacy, and autonomy.

In this paper, we take a survey on the existing FLSs from a system view. First, we show the definition of FLSs, and compare it with conventional federated systems. Second, we analyze the system components of FLSs, including the parties, the manager, and the computation-communication framework. Third, we categorize FLSs based on six different aspects: data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation, and motivation of federation. These aspects can direct the design of an FLS as common building blocks and system abstractions. Fourth, based on these aspects, we systematically summarize the existing studies, which can be used to direct the design of FLSs. Last, to make FL more practical and powerful, we present future research directions to work on. We believe that systems and infrastructures are essential for the success of FL. More work has to be carried out to address the system research issues in effectiveness, efficiency, privacy, and autonomy.

1.1 Related Surveys

There have been several surveys on FL. A seminal survey written by Yang et al. [207] introduces the basics and concepts in FL, and further proposes a comprehensive secure FL framework. The paper mainly target at a relatively small number of parties which are typically enterprise data owners. Li et al. [109] summarize challenges and future directions of FL in massive networks of mobile and edge devices. Recently, Kairouz et al. [85] have a comprehensive description about the characteristics and challenges on FL from different research topics. However, they mainly focus on cross-device FL, where the participants are a very large number of mobile or IoT devices. More recently, another survey [11] summarizes the platforms, protocols and applications of federated learning. Some surveys only focus on an aspect of federated learning. For example, Lim et al. [113] conduct a survey of FL specific to mobile edge computing, while [125] focuses on the threats to federated learning.

1.2 Our Contribution

To the best of our knowledge, there lacks a survey on reviewing existing systems and infrastructure of FLSs and on boosting the attention of creating systems for FL (Similar to prosperous system research in deep learning). In comparison with the previous surveys, the main contributions of this paper are as follows. (1) Our survey is the first one to provide a comprehensive analysis on FL from a system’s point of view, including system components, taxonomy, summary, design, and vision. (2) We provide a comprehensive taxonomy against FLSs on six different aspects, including data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation, and motivation of federation, which can be used as common building blocks and system abstractions of FLSs. (3) We summarize existing typical and state-of-the-art studies according to their domains, which is convenient for researchers and developers to refer to. (4) We present the design factors for a successful FLS and comprehensively review solutions for each scenario. (5) We propose interesting research directions and challenges for future generations of FLSs.

The rest of the paper is organized as follows. In Section 2, we introduce the concept and the system components of FLSs. In Section 3, we propose six aspects to classify FLSs. In Section 4, we summary existing studies and systems on FL. We then present the design factors and solutions for an FLS in Section 5. Last, we propose possible future directions on FL in Section 7 and conclude our paper in Section 8.

2 An Overview of Federated Learning Systems

2.1 Background

As data breach becomes a major concern, more and more governments establish regulations to protect users’ data, such as GDPR in European Union [185], PDPA in Singapore [39], and CCPA [1] in the US. The cost of breaching these policies is pretty high for companies. In a breach of 600,000 drivers’ personal information in 2016, Uber had to pay 148 million to settle the investigation [[3](#bib.bib3)]. SingHealth was fined 750,000 by the Singapore government for a breach of PDPA [5]. Google was fined $57 million for a breach of GDPR [4], which is the largest penalty as of March 18, 2020 under the European Union privacy law.

Under the above circumstances, federated learning, a collaborative learning without exchanging users’ original data, has drawn increasingly attention nowadays. While machine learning, especially deep learning, has attracted many attentions again recently, the combination of federation and machine learning is emerging as a new and hot research topic.

2.2 Definition

FL enables multiple parties jointly train a machine learning model without exchanging the local data. It covers the techniques from multiple research areas such as distributed system, machine learning, and privacy. Inspired by the definition of FL given by other studies [85, 207], here we give a definition of FLSs.

In a federated learning system, multiple parties collaboratively train machine learning models without exchanging their raw data. The output of the system is a machine learning model for each party (which can be same or different). A practical federated learning system has the following constraint: given an evaluation metric such as test accuracy, the performance of the model learned by federated learning should be better than the model learned by local training with the same model architecture.

2.3 Compare with Conventional Federated Systems

The concept of federation can be found with its counterparts in the real world such as business and sports. The main characteristic of federation is cooperation. Federation not only commonly appears in society, but also plays an important role in computing. In computer science, federated computing systems have been an attractive area of research under different contexts.

Around 1990, there were many studies on federated database systems (FDBSs) [166]. An FDBS is a collection of autonomous databases cooperating for mutual benefits. As pointed out in a previous study [166], three important components of an FDBS are autonomy, heterogeneity, and distribution.

• •

Autonomy. A database system (DBS) that participates in an FDBS is autonomous, which means it is under separate and independent control. The parties can still manage the data without the FDBS.

• •

Heterogeneity. The database management systems can be different inside an FDBS. For example, the difference can lie in the data structures, query languages, system software requirements, and communication capabilities.

• •

Distribution. Due to the existence of multiple DBSs before an FDBS is built, the data distribution may differ in different DBSs. A data record can be horizontally or vertically partitioned into different DBSs, and can also be duplicated in multiple DBSs to increase the reliability.

More recently, with the development of cloud computing, many studies have been done for federated cloud computing [97]. A federated cloud (FC) is the deployment and management of multiple external and internal cloud computing services. The concept of cloud federation enables further reduction of costs due to partial outsourcing to more cost-efficient regions. Resource migration and resource redundancy are two basic features of federated clouds [97]. First, resources may be transferred from one cloud provider to another. Migration enables the relocation of resources. Second, redundancy allows concurrent usage of similar service features in different domains. For example, the data can be partitioned and processed at different providers following the same computation logic. Overall, the scheduling of different resources is a key factor in the design of a federated cloud system.

There are some similarities and differences between FLSs and conventional federated systems. First, the concept of federation still applies. The common and basic idea is about the cooperation of multiple independent parties. Therefore, the perspective of considering heterogeneity and autonomy among the parties can still be applied to FLSs. Second, some factors in the design of distributed systems are still important for FLSs. For example, how the data are shared between the parties can influence the efficiency of the systems. For the differences, these federated systems have different emphasis on collaboration and constraints. While FDBSs focus on the management of distributed data and FCs focus on the scheduling of the resources, FLSs care more about the secure computation among multiple parties. FLSs induce new challenges such as the algorithm designs of the distributed training and the data protection under the privacy restrictions.

Figure 1 shows the number of papers in each year for these three research areas. Here we count the papers by searching keywords “federated database”, “federated cloud”, and “federated learning” in Google Scholar111https://scholar.google.com/. Although federated database was proposed 30 years ago, there are still about 400 papers that mentioned it in recent years. The popularity of federated cloud grows more quickly than federated database at the beginning, while it appears to decrease in recent years probably because cloud computing becomes more mature and the incentives of federation diminish. For FL, the number of related papers is increasing rapidly and has achieved about 4,400 last year. Nowadays, the “data island” phenomena are common and have increasingly become an important issue in machine learning. Also, there is a increasing privacy concern and social awareness from the general public. Thus, we expect the popularity of FL will keep increasing for at least five years until there may be mature FLSs.

2.4 System Components

There are three major components in an FLS: parties (e.g., clients), the manager (e.g., server), and the communication-computation framework to train the machine learning model.

2.4.1 Parties

In FLSs, the parties are the data owners and the beneficiaries of FL. They can be organizations or mobile devices, named cross-silo or cross-device settings [85], respectively. We consider the following properties of the parties that affect the design of FLSs.

First, what is the hardware capacity of the parties? The hardware capacity includes the computation power and storage. If the parties are mobile phones, the capacity is weak and the parties cannot perform much computation and train a large model. For example, Wang et al. [192] consider a resource constrained setting in FL. They design an objective to include the resource budget and proposed an algorithm to determine the rounds of local updates.

Second, what is the scale and stability of the parties? For organizations, the scale is relative small compared with the mobile devices. Also, the stability of the cross-silo setting is better than the cross-device setting. Thus, in the cross-silo setting, we can expect that every party can continuously conduct computation and communication tasks in the entire federated process, which is a common setting in many studies [104, 38, 169]. If the parties are mobile devices, the system has to handle possible issues such as connection lost [24]. Moreover, since the number of devices can be very large (e.g., millions), it is unpractical to assume all the devices to participate every round in FL. The widely used setting is to choose a fraction of devices to perform computation in each round [129, 24].

Last, what are the data distributions among the parties? Usually, no matter cross-device or cross-silo setting, the non-IID (identically and independently distributed) data distribution is considered a practical and challenging setting in federated learning [85], which is evaluated in the experiments of recent work [104, 213, 111, 189]. Such non-IID data distribution may be more obvious among the organizations. For example, a bank and an insurance company can conduct FL to improve their predictions (e.g., whether a person can repay the loan and whether the person will buy the insurance products), while even the features can vary a lot in these organizations. Techniques in transfer learning [147], meta-learning [55], and multi-task learning [157] may be useful to combine the knowledge of various kinds of parties.

2.4.2 Manager

In the cross-device setting, the manager is usually a powerful central server. It conducts the training of the global machine learning model and manages the communication between the parties and the server. The stability and reliability of the server are quite important. Once the server fails to provide the accurate computation results, the FLS may produce a bad model. To address these potential issues, blockchain [176] may be a possible technique to offer a decentralized solution in order to increase the system reliability. For example, Kim et al. [93] leverage the blockchain in lieu of the central server in their system, where the blockchain enables exchanging the devices’ updates and providing rewards to them.

In the cross-silo setting, since the organizations are expected to have powerful machines, the manager can also be one of the organizations who dominates the FL process. This is particularly used in the vertical FL [207], which we will introduce in Section 3.1 in detail. In a vertical FL setting by Liu et al. [119], the features of data are vertically partitioned across the parties and only one party has the labels. The party that owns the labels is naturally considered as the FL manager.

One challenge can be that it is hard to find a trusted server or party as the manager, especially in the cross-silo setting. Then, a fully-decentralized setting can be a good choice, where the parties communicate with each other directly and almost equally contribute to the global machine learning model training. These parties jointly set a FL task and deploy the FLS. Li et al. [104] propose a federated gradient boosting decision trees framework, where each party trains decision trees sequentially and the final model is the combination of all trees. It is challenging to design a fully-decentralized FLS with reasonable communication overhead.

2.4.3 Communication-Computation Framework

In FLSs, the computation happens on the parties and the manager, while the communication happens between the parties and the manager. Usually, the aim of the computation is for the model training and the aim of the communication is for exchanging the model parameters.

A basic and widely used framework is Federated Averaging (FedAvg) [129] proposed in 2016, as shown in Figure 2a. In each iteration, the server first sends the current global model to the selected parties. Then, the selected parties update the global model with their local data. Next, the updated models are sent back to the server. Last, the server averages all the received local models to get a new global model. FedAvg repeats the above process until reaching the specified number of iterations. The global model of the server is the final output.

While FedAvg is a centralized FL framework, SimFL, proposed by Li et al. [109], represents a decentralized FL framework. In SimFL, no trusted server is needed. In each iteration, the parties first update the gradients of their local data. Then, the gradients are sent to a selected party. Next, the selected party use its local data and the gradients to update the model. Last, the model is sent to all the other parties. To ensure fairness and utilize the data from different parties, every party is selected for updating the model for about the same number of rounds. SimFL repeats a specified number of iterations and outputs the final model.

3 Taxonomy

Considering the common system abstractions and building blocks for different FLSs, we classify FLSs by six aspects: data partitioning, machine learning model, privacy mechanism, communication architecture, scale of federation, and motivation of federation. These aspects include common factors (e.g., data partitioning, communication architecture) in previous FLSs [166, 97] and unique consideration (e.g., machine learning model and privacy mechanism) for FLSs. Furthermore, these aspects can be used to guide the design of FLSs. Figure 3 shows the summary of the taxonomy of FLSs.

In Table 1 of [85], they consider different characteristics to distinguish distributed learning, cross-device federated learning, and cross-silo federated learning, including setting, data distribution, communication, etc. Our taxonomy is used to distinguish different federated learning systems from a deployment view, and aspects like machine learning models and motivation of federation are not considered in [85].

3.1 Data Partitioning

Based on how data are distributed over the sample and feature spaces, FLSs can be typically categorized in horizontal, vertical, and hybrid FLSs [207].

In horizontal FL, the datasets of different parties have the same feature space but little intersection on the sample space. This is a natural data partitioning especially for the cross-device setting, where different users try to improve their model performance on the same task using FL. Also, the majority of FL studies adopt horizontal partitioning. Since the local data are in the same feature space, the parties can train the local models using their local data with the same model architecture. The global model can simply be updated by averaging all the local models. A basic and popular framework of horizontal federated learning is FedAvg, as shown in Figure 2. Wake-word recognition [98], such as ‘Hey Siri’ and ‘OK Google’, is a typical application of horizontal partition because each user speaks the same sentence with a different voice.

In vertical FL, the datasets of different parties have the same or similar sample space but differ in the feature space. For the vertical FLS, it usually adopts entity alignment techniques [206, 41] to collect the overlapped samples of the parties. Then the overlapped data are used to train the machine learning model using encryption methods. Cheng et al. [38] propose a lossless vertical FLS to enable parties to collaboratively train gradient boosting decision trees. They use privacy-preserving entity alignment to find common users among two parties, whose gradients are used to jointly train the decision trees. Cooperation among different companies usually can be treated as a situation of vertical partition.

In many other applications, while existing FLSs mostly focus on one kind of partition, the partition of data among the parties may be a hybrid of horizontal partition and vertical partition. Let us take cancer diagnosis system as an example. A group of hospitals wants to build an FLS for cancer diagnosis but each hospital has different patients as well as different kinds of medical examination results. Transfer learning [147] is a possible solution for such scenarios. Liu et al. [119] propose a secure federated transfer learning system which can learn a representation among the features of parties using common instances.

3.2 Machine Learning Models

Since FL is used to solve machine learning problems, the parties usually want to train a state-of-the-art machine learning model on a specified task. There have been many efforts in developing new models or reinventing current models to the federated setting. Here, we consider the widely-used models nowadays. The most popular machine learning model now is neural network (NN), which achieves state-of-the-art results in many tasks such as image classification and word prediction [96, 175]. There are many federated learning studies based on stochastic gradient descent [129, 189, 24], which can be used to train NNs.

Another widely used model is decision tree, which is highly efficient to train and easy to interpret compared with NNs. A tree-based FLS is designed for the federated training of single or multiple decision trees (e.g., gradient boosting decision trees (GBDTs) and random forests). GBDTs are especially popular recently and it has a very good performance in many classification and regression tasks. Li et al. [104] and Cheng et al. [38] propose FLSs for GBDTs on horizontally and vertically partitioned data, respectively.

Besides NNs and trees, linear models (e.g., linear regression, logistic regression, SVM) are classic and easy-to-use models. There are some well developed systems for linear regression and logistic regression [141, 72]. These linear models are easy to learn compared with other complex models (e.g., NNs).

While a single machine learning model may be weak, ensemble methods [150] such as stacking and voting can be applied in the federated setting. Each party trains a local model and sends it to the server, which aggregates all the models as an ensemble. The ensemble can directly be used for prediction by max voting or be used to train a meta-model by stacking. A benefit of federated ensemble learning is that each party can train heterogeneous models as there is no averaging of model parameters. As shown in previous studies [213, 103], federated ensemble learning can also achieve a good accuracy in a single communication round.

Currently, many FL frameworks [129, 94, 192, 108] are proposed based on stochastic gradient descent, which is a typical optimization algorithm for many models including neural networks and logistic regression. However, to increase the effectiveness of FL, we may have to exploit the model architecture [189]. Since the research of FL is still at an early stage, there is still a gap for FLSs to better support the state-of-the-art models.

3.3 Privacy Mechanisms

Although the local data are not exposed in FL, the exchanged model parameters may still leak sensitive information about the data. There have been many attacks against machine learning models [56, 167, 137, 131], such as model inversion attack [56] and membership inference attack [167], which can potentially infer the raw data by accessing to the model. Moreover, there are many privacy mechanisms such as differential privacy [48] and $k$-anonymity [50], which provide different privacy guarantees. The characteristics of existing privacy mechanisms are summarized in the survey [186]. Here we introduce two major approaches that are adopted in the current FLSs for data protection: cryptographic methods and differential privacy.

Cryptographic methods such as homomorphic encryption  [15, 72, 28, 156, 116], and secure multi-party computation (SMC) [165, 32, 22, 23] are widely used in privacy-preserving machine learning algorithms. Basically, the parties have to encrypt their messages before sending, operate on the encrypted messages, and decrypt the encrypted output to get the final result. Applying the above methods, the user privacy of FLSs can usually be well protected [89, 211, 91, 144]. For example, SMC [63] guarantees that all the parties cannot learn anything except the output, which can be used to securely aggregate the transferred gradients. However, SMC does not provide privacy guarantees for the final model, which is still vulnerable to the inference attacks and model inversion attacks [167, 56]. Also, due to the additional encryption and decryption operations, such systems suffer from the extremely high computation overhead.

Differential privacy [48, 49] guarantees that one single record does not influence much on the output of a function. Many studies adopt differential privacy [31, 8, 105, 180, 220, 112] for data privacy protection to ensure the parties cannot know whether an individual record participates in the learning or not. By injecting random noises to the data or model parameters [8, 105, 170, 202], differential privacy provides statistical privacy guarantees for individual records and protection against the inference attack on the model. Due to the noises in the learning process, such systems tend to produce less accurate models.

Note that the above methods are independent of each other, and an FLS can adopt multiple methods to enhance the privacy guarantees [64, 205, 86]. There are also other approaches to protect the user privacy. An interesting hardware-based approach is to use trusted execution environment (TEE) such as Intel SGX processors [159, 145], which can guarantee that the code and data loaded inside are protected. Such environment can be used inside the central server to increase its credibility.

Related to privacy level, the threat models also vary in FLSs [125]. The attacks can come from any stage of the process of FL, including inputs, the learning process, and the learnt model.

• •

Inputs The malicious parties can conduct data poisoning attacks [35, 99, 12] on FL. For example, the parties can modify the labels of training samples with a specific class, so that the final model performs badly on this class.

• •

Learning process During the learning process, the parties can perform model poisoning attacks [16, 203] to upload designed model parameters. Like data poisoning attacks, the global model can have a very low accuracy due to the poisoned local updating. Besides model poisoning attacks, the Byzantine fault [27, 21, 173] is also a common issue in distributed learning, where the parties may behave arbitrarily badly and upload random updates.

• •

The learnt model. If the learnt model is published, the inference attacks [56, 167, 131, 137] can be conducted on it. The server can infer sensitive information about the training data from the exchanged model parameters. For example, membership inference attacks [167, 137] can infer whether a specific data record is used in the training. Note that the inference attacks may also be conducted in the learning process by the FL manager, who has access to the local updates of the parties.

3.4 Communication Architecture

There are two major ways of communication in FLSs: centralized design and decentralized design. In the centralized design, the data flow is often asymmetric, which means the manager aggregates the information (e.g., local models) from parties and sends back training results [24]. The parameter updates on the global model are always done in this manager. The communication between the manager and the local parties can be synchronous [129] or asynchronous [204, 171]. In a decentralized design, the communications are performed among the parties [217, 104] and every party is able to update the global parameters directly.

Google Keyboard [71] is a case of centralized architecture. The server collects local model updates from users’ devices and trains a global model, which is sent back to the users for inference, as shown in Figure 2a. The scalability and stability are two important factors in the system design of the centralized FL.

While the centralized design is widely used in existing studies, the decentralized design is preferred at some aspects since concentrating information on one server may bring potential risks or unfairness. However, the design of the decentralized communication architecture is challenging, which should take fairness and communication overhead into consideration. There are currently three different decentralized designs: P2P [104, 217], graph [128] or blockchain [191, 219]. In a P2P design, the parties are equally privileged and treated during federated learning. An example is SimFL [104], where each party trains a tree sequentially and sends the tree to all the other parties. The communication architecture can also be modeled as a graph with the additional constrains such as latency and computation time. Marfoq et al. [128] propose an algorithm to find a throughput-optimal topology design. Recently, blockchain [223] is a popular decentralized platform for consideration. It can be used to store the information of parties in federated learning and ensure the transparency of federated learning [191].

3.5 Scale of Federation

The FLSs can be categorized into two typical types by the scale of federation: cross-silo FLSs and cross-device FLSs [85]. The differences between them lie on the number of parties and the amount of data stored in each party.

In cross-silo FLS, the parties are organizations or data centers. There are usually a relatively small number of parties and each of them has a relatively large amount of data as well as computational power. For example, Amazon wants to recommend items for users by training the shopping data collected from hundreds of data centers around the world. Each data center possesses a huge amount of data as well as sufficient computational resources. Another example is that federated learning can be used among medical institutions. Different hospitals can use federated learning to train a CNN for chest radiography classification while keeping their chest X-ray images locally [86]. With federated learning, the accuracy of the model can be significantly improved. One challenge that such FLS faces is how to efficiently distribute computation to data centers under the constraint of privacy models [224].

In cross-device FLS, on the contrary, the number of parties is relatively large and each party has a relatively small amount of data as well as computational power. The parties are usually mobile devices. Google Keyboard [208] is an example of cross-device FLSs. The query suggestions of Google Keyboard can be improved with the help of FL. Due to the energy consumption concern, the devices cannot be asked to conduct complex training tasks. Under this occasion, the system should be powerful enough to manage a large number of parties and deal with possible issues such as the unstable connection between the device and the server.

3.6 Motivation of Federation

In real-world applications of FL, individual parties need the motivation to get involved in the FLS. The motivation can be regulations or incentives. FL inside a company or an organization is usually motivated by regulations (e.g., FL across different departments of a company). For example, the department which has the transaction records of users can help another department to predict user credit by federated learning. In many cases, parties cannot be forced to provide their data by regulations. However, parties that choose to participate in federated learning can benefit from it, e.g., higher model accuracy. For example, hospitals can conduct federated learning to train a machine learning model for chest radiography classification [86] or COVID-19 detection [152]. Then, the hospitals can get a good model which has a higher accuracy than human experts and the model trained locally without federation. Another example is Google Keyboard [208]. While users have the choice to prevent Google Keyboard from utilizing their data, those who agree to upload input data may enjoy a higher accuracy of word prediction. Users may be willing to participate in federated learning for their convenience.

A challenging problem is how to design a fair incentive mechanism, such that the party that contributes more can also benefit more from federated learning. There have been some successful cases for incentive designs in blockchain [228, 51]. The parties inside the system can be collaborators as well as competitors. Other incentive designs like [88, 87] are proposed to attract participants with high-quality data for FL. We expect game theory models [163, 84, 136] and their equilibrium designs should be revisited under the FLSs. Even in the case of Google Keyboard, the users need to be motivated to participate this collaborative learning process.

4 Summary of Existing Studies

In this section222Last updated on . We will periodically update this section to include the state-of-the-art and valuable FL studies. Please check out our latest version at this URL: https://arxiv.org/abs/1907.09693. Also, if you have any reference that you want to add into this survey, kindly drop Dr. Bingsheng He an email (hebs@comp.nus.edu.sg)., we summarize and compare the existing studies on FLSs according to the aspects considered in Section 3.

4.1 Methodology

To discover the existing studies on FL, we search keyword “Federated Learning” in Google Scholar. Here we only consider the published studies in the computer science community.

Since the scale of federation and the motivation of federation are problem dependent, we do not compare the existing studies by these two aspects. For ease of presentation, we use “NN”, “DT” and “LM” to denote neural networks, decision trees and linear models, respectively. Moreover, we use “CM” and “DP” to denote cryptographic methods and differential privacy, respectively. Note that the algorithms (e.g., federated stochastic gradient descent) in some studies can be used to learn many machine learning models (e.g., logistic regression and neural networks). Thus, in the “model implementation” column, we present the models implemented in the experiments of corresponding papers. Moreover, in the “main area” column, we indicate the major area that the papers study on.

4.2 Individual Studies

We summarize existing popular and state-of-the-art research work, as shown in Table 4.2. From Table 4.2, we have the following four key findings.

First, most of the existing studies consider a horizontal data partitioning. We conjecture a part of the reason is that the experimental studies and benchmarks in horizontal data partitioning are relatively ready than vertical data partitioning. However, vertical FL is also common in real world, especially between different organizations. Vertical FL can enable more collaboration between diverse parties. Thus, more efforts should be paid to vertical FL to fill the gap.

Second, most studies consider exchanging the raw model parameters without any privacy guarantees. This may not be right if more powerful attacks on machine learning models are discovered in the future. Currently, the mainstream methods to provide privacy guarantees are differential privacy and cryptographic methods (e.g., secure multi-party computation and homomorphic encryption). Differential privacy may influence the final model quality a lot. Moreover, the cryptographic methods bring much computation and communication overhead and may be the bottleneck of FLSs. We look forward to a cheap way with reasonable privacy guarantees to satisfy the regulations.

Third, the centralized design is the mainstream of current implementations. A trusted server is needed in their settings. However, it may be hard to find a trusted server especially in the cross-silo setting. One naive approach to remove the central server is that the parties share the model parameters with all the other parties and each party also maintains the same global model locally. This method bring more communication and computation cost compared with the centralized setting. More studies should be done for practical FL with the decentralized architecture.

Last, the main research directions (also the main challenge) of FL are to improve the effectiveness, efficiency, and privacy, which are also three important metrics to evaluate an FLS. Meanwhile, there are many other research topics on FL such as fairness and incentive mechanisms. Since FL is related to many research areas, we believe that FL will attract more researchers and we can see more interesting studies in the near future.