Exact Unlearning from Proxies Induces Closeness Guarantees on Approximate Unlearning

TL;DR

This paper introduces a novel approach to machine unlearning that leverages data distribution structures for exact unlearning, providing theoretical guarantees and empirical validation across multiple scenarios.

Contribution

It establishes a new paradigm linking unlearning to data distributions, with theoretical bounds and practical algorithms for achieving exact unlearning.

Findings

Theoretical bounds on KL divergence demonstrate framework soundness.

Experimental results show closest classifier to ideal retraining.

Method outperforms existing competitors in forgetting scenarios.

Abstract

This paper proposes a paradigm shift linking machine unlearning directly to the structure of the data distributions rather than a mere update of the neural network parameters. We show that inferring these distributions with precision enables distilling the exact unlearning signal induced by the modeling. Theoretical bounds on the Kullback-Leibler divergence from the ideal retrained model to our unlearned model, under verifiable admissibility criterion, reveal the soundness of our framework. This method is experimentally validated over three forgetting scenarios as reaching the closest classifier to the ideal retrained model when compared to competitors.