A General Framework to Enhance Fine-tuning-based LLM Unlearning

TL;DR

This paper introduces GRUN, a novel framework that enhances the utility of fine-tuning-based unlearning in LLMs by unifying and improving existing methods, leading to better data removal and model performance.

Contribution

The paper proposes Gated Representation UNlearning (GRUN), a general, efficient framework that improves unlearning effectiveness and utility in fine-tuning-based LLM unlearning methods.

Findings

GRUN significantly improves unlearning accuracy.

GRUN maintains higher model utility after unlearning.

The framework is effective across different unlearning scenarios.

Abstract

Unlearning has been proposed to remove copyrighted and privacy-sensitive data from Large Language Models (LLMs). Existing approaches primarily rely on fine-tuning-based methods, which can be categorized into gradient ascent-based (GA-based) and suppression-based methods. However, they often degrade model utility (the ability to respond to normal prompts). In this work, we aim to develop a general framework that enhances the utility of fine-tuning-based unlearning methods. To achieve this goal, we first investigate the common property between GA-based and suppression-based methods. We unveil that GA-based methods unlearn by distinguishing the target data (i.e., the data to be removed) and suppressing related generations, which is essentially the same strategy employed by suppression-based methods. Inspired by this finding, we introduce Gated Representation UNlearning (GRUN) which has two components: a soft gate function for distinguishing target data and a suppression module using Representation Fine-tuning (ReFT) to adjust representations rather than model parameters. Experiments show that GRUN significantly improves the unlearning and utility. Meanwhile, it is general for fine-tuning-based methods, efficient and promising for sequential unlearning.