Unlearning in LLMs: Methods, Evaluation, and Open Challenges

TL;DR

This paper surveys methods for unlearning in large language models, evaluating current approaches, benchmarks, and challenges to guide future research on responsible model management.

Contribution

It provides a comprehensive categorization of unlearning techniques, reviews evaluation tools, and outlines open challenges in the field of LLM unlearning.

Findings

Categorizes unlearning methods into data-centric, parameter-centric, architecture-centric, and hybrid approaches.

Reviews existing benchmarks and metrics for evaluating unlearning effectiveness.

Identifies key open challenges such as scalability, formal guarantees, and robustness.

Abstract

Large language models (LLMs) have achieved remarkable success across natural language processing tasks, yet their widespread deployment raises pressing concerns around privacy, copyright, security, and bias. Machine unlearning has emerged as a promising paradigm for selectively removing knowledge or data from trained models without full retraining. In this survey, we provide a structured overview of unlearning methods for LLMs, categorizing existing approaches into data-centric, parameter-centric, architecture-centric, hybrid, and other strategies. We also review the evaluation ecosystem, including benchmarks, metrics, and datasets designed to measure forgetting effectiveness, knowledge retention, and robustness. Finally, we outline key challenges and open problems, such as scalable efficiency, formal guarantees, cross-language and multimodal unlearning, and robustness against adversarial relearning. By synthesizing current progress and highlighting open directions, this paper aims to serve as a roadmap for developing reliable and responsible unlearning techniques in large language models.