A hybrid framework for effective and efficient machine unlearning

TL;DR

This paper introduces a hybrid machine unlearning framework that balances accuracy and efficiency by adaptively choosing between retraining and parameter modification, significantly improving unlearning performance.

Contribution

It proposes a novel hybrid strategy combining exact and approximate unlearning methods with an adaptive selection mechanism for better efficiency and accuracy.

Findings

Unlearning efficiency improved by 1.5× to 8×

Achieves comparable accuracy to retraining from scratch

Effectively balances accuracy and computational cost

Abstract

Recently machine unlearning (MU) is proposed to remove the imprints of revoked samples from the already trained model parameters, to solve users' privacy concern. Different from the runtime expensive retraining from scratch, there exist two research lines, exact MU and approximate MU with different favorites in terms of accuracy and efficiency. In this paper, we present a novel hybrid strategy on top of them to achieve an overall success. It implements the unlearning operation with an acceptable computation cost, while simultaneously improving the accuracy as much as possible. Specifically, it runs reasonable unlearning techniques by estimating the retraining workloads caused by revocations. If the workload is lightweight, it performs retraining to derive the model parameters consistent with the accurate ones retrained from scratch. Otherwise, it outputs the unlearned model by directly modifying the current parameters, for better efficiency. In particular, to improve the accuracy in the latter case, we propose an optimized version to amend the output model with lightweight runtime penalty. We particularly study the boundary of two approaches in our frameworks to adaptively make the smart selection. Extensive experiments on real datasets validate that our proposals can improve the unlearning efficiency by 1.5$\times$ to 8$\times$ while achieving comparable accuracy.