Machine Unlearning for Traditional Models and Large Language Models: A Short Survey

TL;DR

This survey reviews recent advances in machine unlearning, focusing on traditional models and large language models, highlighting evaluation methods, challenges, and future research directions for privacy-preserving machine learning.

Contribution

It provides a comprehensive categorization, evaluation criteria, and analysis of unlearning techniques for both traditional models and LLMs, filling a gap in current literature.

Findings

Current unlearning methods have notable limitations.

Evaluation standards for unlearning effectiveness are proposed.

Future research directions are identified for improving unlearning techniques.

Abstract

With the implementation of personal data privacy regulations, the field of machine learning (ML) faces the challenge of the "right to be forgotten". Machine unlearning has emerged to address this issue, aiming to delete data and reduce its impact on models according to user requests. Despite the widespread interest in machine unlearning, comprehensive surveys on its latest advancements, especially in the field of Large Language Models (LLMs) is lacking. This survey aims to fill this gap by providing an in-depth exploration of machine unlearning, including the definition, classification and evaluation criteria, as well as challenges in different environments and their solutions. Specifically, this paper categorizes and investigates unlearning on both traditional models and LLMs, and proposes methods for evaluating the effectiveness and efficiency of unlearning, and standards for performance measurement. This paper reveals the limitations of current unlearning techniques and emphasizes the importance of a comprehensive unlearning evaluation to avoid arbitrary forgetting. This survey not only summarizes the key concepts of unlearning technology but also points out its prominent issues and feasible directions for future research, providing valuable guidance for scholars in the field.