FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method

TL;DR

FedUHB introduces an exact federated unlearning method using the Polyak heavy ball technique, significantly improving efficiency and resource conservation while ensuring complete data removal in federated learning.

Contribution

The paper presents FedUHB, a novel unlearning approach that achieves exact data removal with faster retraining and a dynamic stopping mechanism, advancing federated unlearning methods.

Findings

FedUHB outperforms existing methods in unlearning speed.

It maintains high model accuracy after unlearning.

The dynamic stopping reduces iteration count and resource use.

Abstract

Federated learning facilitates collaborative machine learning, enabling multiple participants to collectively develop a shared model while preserving the privacy of individual data. The growing importance of the "right to be forgotten" calls for effective mechanisms to facilitate data removal upon request. In response, federated unlearning (FU) has been developed to efficiently eliminate the influence of specific data from the model. Current FU methods primarily rely on approximate unlearning strategies, which seek to balance data removal efficacy with computational and communication costs, but often fail to completely erase data influence. To address these limitations, we propose FedUHB, a novel exact unlearning approach that leverages the Polyak heavy ball optimization technique, a first-order method, to achieve rapid retraining. In addition, we introduce a dynamic stopping mechanism to optimize the termination of the unlearning process. Our extensive experiments show that FedUHB not only enhances unlearning efficiency but also preserves robust model performance after unlearning. Furthermore, the dynamic stopping mechanism effectively reduces the number of unlearning iterations, conserving both computational and communication resources. FedUHB can be proved as an effective and efficient solution for exact data removal in federated learning settings.