Federated Unlearning with Knowledge Distillation

TL;DR

This paper introduces a federated unlearning approach using knowledge distillation that effectively removes a client's influence from the model without requiring client data or participation, ensuring privacy and compliance with data removal requests.

Contribution

It presents a novel unlearning method for federated learning that subtracts client updates and uses knowledge distillation, applicable to any neural network type and independent of client involvement.

Findings

Effective removal of client data influence demonstrated on multiple datasets

Method maintains model performance after unlearning

Efficient and practical for real-world federated systems

Abstract

Federated Learning (FL) is designed to protect the data privacy of each client during the training process by transmitting only models instead of the original data. However, the trained model may memorize certain information about the training data. With the recent legislation on right to be forgotten, it is crucially essential for the FL model to possess the ability to forget what it has learned from each client. We propose a novel federated unlearning method to eliminate a client's contribution by subtracting the accumulated historical updates from the model and leveraging the knowledge distillation method to restore the model's performance without using any data from the clients. This method does not have any restrictions on the type of neural networks and does not rely on clients' participation, so it is practical and efficient in the FL system. We further introduce backdoor attacks in the training process to help evaluate the unlearning effect. Experiments on three canonical datasets demonstrate the effectiveness and efficiency of our method.