A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications

TL;DR

This survey reviews federated transfer learning, highlighting its challenges, methods, and applications, emphasizing how it addresses data heterogeneity, privacy, and system issues in distributed machine learning.

Contribution

It provides a comprehensive categorization and review of current federated transfer learning techniques, solutions, datasets, and applications, filling a gap in synthesized knowledge.

Findings

Identifies key challenges in federated transfer learning.

Summarizes existing solutions and methods.

Outlines common datasets and applications.

Abstract

Federated learning (FL) is a novel distributed machine learning paradigm that enables participants to collaboratively train a centralized model with privacy preservation by eliminating the requirement of data sharing. In practice, FL often involves multiple participants and requires the third party to aggregate global information to guide the update of the target participant. Therefore, many FL methods do not work well due to the training and test data of each participant may not be sampled from the same feature space and the same underlying distribution. Meanwhile, the differences in their local devices (system heterogeneity), the continuous influx of online data (incremental data), and labeled data scarcity may further influence the performance of these methods. To solve this problem, federated transfer learning (FTL), which integrates transfer learning (TL) into FL, has attracted the attention of numerous researchers. However, since FL enables a continuous share of knowledge among participants with each communication round while not allowing local data to be accessed by other participants, FTL faces many unique challenges that are not present in TL. In this survey, we focus on categorizing and reviewing the current progress on federated transfer learning, and outlining corresponding solutions and applications. Furthermore, the common setting of FTL scenarios, available datasets, and significant related research are summarized in this survey.