On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions

TL;DR

This paper surveys recent advances in knowledge editing within federated learning, including methods for knowledge augmentation and removal, and introduces the concept of Federated Editable Learning to unify the field.

Contribution

It provides a comprehensive survey of knowledge editing techniques in federated learning and proposes the FEL paradigm to unify and evaluate these methods.

Findings

Overview of existing knowledge editing methods in FL

Evaluation of methods within the FEL framework

Identification of current challenges and future research directions

Abstract

As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall framework when learning over a sequence of tasks results in the phenomenon known as ``catastrophic forgetting''. Consequently, much FL research has centered on devising federated increasing learning methods to alleviate forgetting while augmenting knowledge. On the other hand, forgetting is not always detrimental. The selective amnesia, also known as federated unlearning, which entails the elimination of specific knowledge, can address privacy concerns and create additional ``space'' for acquiring new knowledge. However, there is a scarcity of extensive surveys that encompass recent advancements and provide a thorough examination of this issue. In this manuscript, we present an extensive survey on the topic of knowledge editing (augmentation/removal) in Federated Learning, with the goal of summarizing the state-of-the-art research and expanding the perspective for various domains. Initially, we introduce an integrated paradigm, referred to as Federated Editable Learning (FEL), by reevaluating the entire lifecycle of FL. Secondly, we provide a comprehensive overview of existing methods, evaluate their position within the proposed paradigm, and emphasize the current challenges they face. Lastly, we explore potential avenues for future research and identify unresolved issues.