Robust LLM Unlearning with MUDMAN: Meta-Unlearning with Disruption Masking And Normalization

TL;DR

This paper introduces MUDMAN, a novel method for robustly unlearning dangerous knowledge in language models by combining disruption masking, gradient normalization, and meta-learning, significantly outperforming previous methods.

Contribution

The paper proposes MUDMAN, a new unlearning approach that ensures irreversibility of unlearning in language models through innovative techniques like disruption masking and gradient normalization.

Findings

MUDMAN outperforms TAR by 40% in preventing knowledge recovery.

Disruption masking ensures all updates are non-disruptive.

Gradient normalization enhances unlearning effectiveness.

Abstract

Language models can retain dangerous knowledge and skills even after extensive safety fine-tuning, posing both misuse and misalignment risks. Recent studies show that even specialized unlearning methods can be easily reversed. To address this, we systematically evaluate many existing and novel components of unlearning methods and identify ones crucial for irreversible unlearning. We introduce Disruption Masking, a technique in which we only allow updating weights, where the signs of the unlearning gradient and the retaining gradient are the same. This ensures all updates are non-disruptive. Additionally, we identify the need for normalizing the unlearning gradients, and also confirm the usefulness of meta-learning. We combine these insights into MUDMAN (Meta-Unlearning with Disruption Masking and Normalization) and validate its effectiveness at preventing the recovery of dangerous capabilities. MUDMAN outperforms the prior TAR method by 40%, setting a new state-of-the-art for robust unlearning.