Federated Learning with New Knowledge: Fundamentals, Advances, and Futures

TL;DR

This paper reviews how federated learning systems can incorporate new knowledge like features, tasks, and models to improve efficiency, sustainability, and adaptability, highlighting challenges, methods, and future directions.

Contribution

It systematically defines sources of new knowledge in FL and analyzes methods for integrating them, providing a comprehensive framework and future research directions.

Findings

Identifies key sources of new knowledge in FL.

Analyzes impact of knowledge form and timing.

Discusses future challenges and solutions.

Abstract

Federated Learning (FL) is a privacy-preserving distributed learning approach that is rapidly developing in an era where privacy protection is increasingly valued. It is this rapid development trend, along with the continuous emergence of new demands for FL in the real world, that prompts us to focus on a very important problem: Federated Learning with New Knowledge. The primary challenge here is to effectively incorporate various new knowledge into existing FL systems and evolve these systems to reduce costs, extend their lifespan, and facilitate sustainable development. In this paper, we systematically define the main sources of new knowledge in FL, including new features, tasks, models, and algorithms. For each source, we thoroughly analyze and discuss how to incorporate new knowledge into existing FL systems and examine the impact of the form and timing of new knowledge arrival on the incorporation process. Furthermore, we comprehensively discuss the potential future directions for FL with new knowledge, considering a variety of factors such as scenario setups, efficiency, and security. There is also a continuously updating repository for this topic: https://github.com/conditionWang/FLNK.