Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning

TL;DR

This paper investigates how different subsets of training data, especially syntactically similar queries, affect the effectiveness of unlearning in large language models, revealing the importance of syntactic similarity in privacy-preserving model updates.

Contribution

It introduces the concept of the Syntactically Similar Neighbor Set and demonstrates its significance in LLM unlearning and performance preservation.

Findings

Syntactically similar queries suffer the greatest performance drop during unlearning.

Using this set for regularization improves performance across various data subsets.

Syntactic similarity is more critical than domain or entity relationships in unlearning effectiveness.

Abstract

Large language models (LLMs) risk retaining unauthorized or sensitive information from their training data, which raises privacy concerns. LLM unlearning seeks to mitigate these risks by selectively removing specified data while maintaining overall model performance. However, most existing work focus on methods to achieve effective forgetting and does not provide a detailed analysis of the retain set, the portion of training data that is not targeted for removal. In this paper, we investigate the effects of unlearning on various subsets of the retain set through a case study on entity unlearning. We introduce the Syntactically Similar Neighbor Set, a group of queries that share similar syntactic structures with the data targeted for removal, and show that this subset suffers the greatest performance drop during unlearning. Moreover, when used for regularization, this set not only preserves performance on syntactically similar queries but also delivers comparable or improved results across other data subsets. Our results highlight that syntactic similarity is a critical factor, potentially more so than domain or entity relationships, in achieving effective and practical LLM unlearning.