AdaProb: Efficient Machine Unlearning via Adaptive Probability

TL;DR

AdaProb introduces an efficient, privacy-preserving method for machine unlearning that significantly reduces residual information and computational costs compared to existing approaches.

Contribution

It proposes a novel approach replacing final-layer probabilities with pseudo-probabilities to improve unlearning efficiency and privacy protection.

Findings

Over 20% improvement in forgetting error

Enhanced protection against membership inference attacks

Less than 50% of the computational time compared to state-of-the-art methods

Abstract

Machine unlearning, enabling a trained model to forget specific data, is crucial for addressing erroneous data and adhering to privacy regulations like the General Data Protection Regulation (GDPR)'s "right to be forgotten". Despite recent progress, existing methods face two key challenges: residual information may persist in the model even after unlearning, and the computational overhead required for effective data removal is often high. To address these issues, we propose Adaptive Probability Approximate Unlearning (AdaProb), a novel method that enables models to forget data efficiently and in a privacy-preserving manner. Our method firstly replaces the neural network's final-layer output probabilities with pseudo-probabilities for data to be forgotten. These pseudo-probabilities follow a uniform distribution to maximize unlearning, and they are optimized to align with the model's overall distribution to enhance privacy and reduce the risk of membership inference attacks. Then, the model's weights are updated accordingly. Through comprehensive experiments, our method outperforms state-of-the-art approaches with over 20% improvement in forgetting error, better protection against membership inference attacks, and less than 50% of the computational time.