SoK: Machine Unlearning for Large Language Models

TL;DR

This paper provides a comprehensive survey of machine unlearning in large language models, proposing a new intention-based taxonomy, analyzing evaluation strategies, and discussing practical challenges for deployment.

Contribution

It introduces an intention-oriented taxonomy for unlearning methods, revisits the distinction between true removal and suppression, and evaluates current evaluation metrics and practical challenges.

Findings

Many removal methods may function as suppression rather than true removal

Current evaluation metrics have limitations in assessing unlearning effectiveness

Scalability and sequential unlearning are significant practical challenges

Abstract

Large language model (LLM) unlearning has become a critical topic in machine learning, aiming to eliminate the influence of specific training data or knowledge without retraining the model from scratch. A variety of techniques have been proposed, including Gradient Ascent, model editing, and re-steering hidden representations. While existing surveys often organize these methods by their technical characteristics, such classifications tend to overlook a more fundamental dimension: the underlying intention of unlearning--whether it seeks to truly remove internal knowledge or merely suppress its behavioral effects. In this SoK paper, we propose a new taxonomy based on this intention-oriented perspective. Building on this taxonomy, we make three key contributions. First, we revisit recent findings suggesting that many removal methods may functionally behave like suppression, and explore whether true removal is necessary or achievable. Second, we survey existing evaluation strategies, identify limitations in current metrics and benchmarks, and suggest directions for developing more reliable and intention-aligned evaluations. Third, we highlight practical challenges--such as scalability and support for sequential unlearning--that currently hinder the broader deployment of unlearning methods. In summary, this work offers a comprehensive framework for understanding and advancing unlearning in generative AI, aiming to support future research and guide policy decisions around data removal and privacy.