Learning to Forget using Hypernetworks

TL;DR

HyperForget introduces a hypernetwork-based framework utilizing diffusion models to dynamically generate models that forget specific data, effectively removing targeted information while maintaining overall performance.

Contribution

The paper presents HyperForget, a novel hypernetwork approach combined with diffusion models for effective and adaptive machine unlearning of targeted data.

Findings

Unlearned models achieved zero accuracy on the forget set.

Models retained high accuracy on the retain set.

Demonstrated potential for dynamic targeted data removal.

Abstract

Machine unlearning is gaining increasing attention as a way to remove adversarial data poisoning attacks from already trained models and to comply with privacy and AI regulations. The objective is to unlearn the effect of undesired data from a trained model while maintaining performance on the remaining data. This paper introduces HyperForget, a novel machine unlearning framework that leverages hypernetworks - neural networks that generate parameters for other networks - to dynamically sample models that lack knowledge of targeted data while preserving essential capabilities. Leveraging diffusion models, we implement two Diffusion HyperForget Networks and used them to sample unlearned models in Proof-of-Concept experiments. The unlearned models obtained zero accuracy on the forget set, while preserving good accuracy on the retain sets, highlighting the potential of HyperForget for dynamic targeted data removal and a promising direction for developing adaptive machine unlearning algorithms.