BLUR: A Bi-Level Optimization Approach for LLM Unlearning

TL;DR

This paper introduces BLUR, a bi-level optimization method for unlearning in large language models, prioritizing forgetting specific knowledge while preserving overall utility, and demonstrates its superior performance over existing methods.

Contribution

It formulates the unlearning problem as a bi-level optimization to better balance forgetting and retaining, providing a novel approach with strong theoretical and empirical results.

Findings

BLUR outperforms state-of-the-art unlearning algorithms.

The bi-level formulation effectively balances forget and retain objectives.

Extensive experiments validate the method's superiority across tasks and models.

Abstract

Enabling large language models (LLMs) to unlearn knowledge and capabilities acquired during training has proven vital for ensuring compliance with data regulations and promoting ethical practices in generative AI. Although there are growing interests in developing various unlearning algorithms, it remains unclear how to best formulate the unlearning problem. The most popular formulation uses a weighted sum of forget and retain loss, but it often leads to performance degradation due to the inherent trade-off between forget and retain losses. In this work, we argue that it is important to model the hierarchical structure of the unlearning problem, where the forget problem (which \textit{unlearns} certain knowledge and/or capabilities) takes priority over the retain problem (which preserves model utility). This hierarchical structure naturally leads to a bi-level optimization formulation where the lower-level objective focuses on minimizing the forget loss, while the upper-level objective aims to maintain the model's utility. Based on this new formulation, we propose a novel algorithm, termed Bi-Level UnleaRning (\texttt{BLUR}), which not only possesses strong theoretical guarantees but more importantly, delivers superior performance. In particular, our extensive experiments demonstrate that \texttt{BLUR} consistently outperforms all the state-of-the-art algorithms across various unlearning tasks, models, and metrics. Codes are available at https://github.com/OptimAI-Lab/BLURLLMUnlearning.