Machine Unlearning in Generative AI: A Survey

TL;DR

This survey reviews recent machine unlearning techniques in generative AI, addressing challenges in removing sensitive or biased information from models and discussing evaluation methods and future research directions.

Contribution

It provides a comprehensive overview of machine unlearning in generative AI, including new problem formulations, evaluation approaches, and a structured discussion of existing techniques.

Findings

Identifies challenges in unlearning undesirable knowledge from generative models.

Summarizes various machine unlearning techniques and their advantages and limitations.

Highlights future research directions and open problems in the field.

Abstract

Generative AI technologies have been deployed in many places, such as (multimodal) large language models and vision generative models. Their remarkable performance should be attributed to massive training data and emergent reasoning abilities. However, the models would memorize and generate sensitive, biased, or dangerous information originated from the training data especially those from web crawl. New machine unlearning (MU) techniques are being developed to reduce or eliminate undesirable knowledge and its effects from the models, because those that were designed for traditional classification tasks could not be applied for Generative AI. We offer a comprehensive survey on many things about MU in Generative AI, such as a new problem formulation, evaluation methods, and a structured discussion on the advantages and limitations of different kinds of MU techniques. It also presents several critical challenges and promising directions in MU research. A curated list of readings can be found: https://github.com/franciscoliu/GenAI-MU-Reading.