Certified Unlearning for Neural Networks

TL;DR

This paper introduces a new certified machine unlearning method for neural networks that guarantees removal of specific data influence through noisy fine-tuning, applicable broadly without restrictive assumptions.

Contribution

It proposes a novel unlearning approach based on privacy amplification and stochastic post-processing, providing formal guarantees and broad applicability.

Findings

Achieves provable unlearning guarantees.

Outperforms existing unlearning baselines.

Effective in practical scenarios.

Abstract

We address the problem of machine unlearning, where the goal is to remove the influence of specific training data from a model upon request, motivated by privacy concerns and regulatory requirements such as the "right to be forgotten." Unfortunately, existing methods rely on restrictive assumptions or lack formal guarantees. To this end, we propose a novel method for certified machine unlearning, leveraging the connection between unlearning and privacy amplification by stochastic post-processing. Our method uses noisy fine-tuning on the retain data, i.e., data that does not need to be removed, to ensure provable unlearning guarantees. This approach requires no assumptions about the underlying loss function, making it broadly applicable across diverse settings. We analyze the theoretical trade-offs in efficiency and accuracy and demonstrate empirically that our method not only achieves formal unlearning guarantees but also performs effectively in practice, outperforming existing baselines. Our code is available at https://github.com/stair-lab/certified-unlearning-neural-networks-icml-2025