Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs

TL;DR

This paper reveals that current metrics for evaluating unlearning in LLMs are misleading, as models can be easily reverted through fine-tuning, highlighting the need for representation-level analysis to assess true forgetting.

Contribution

The authors introduce a novel representation-level analysis framework to evaluate the reversibility of unlearning in LLMs, exposing limitations of existing metrics.

Findings

Models can appear to forget but are easily restored via fine-tuning.

Four distinct forgetting regimes are identified based on reversibility and catastrophicity.

Irreversible, non-catastrophic forgetting remains a significant challenge.

Abstract

Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity. We show that these metrics can be misleading, as models can appear to forget while their original behavior is easily restored through minimal fine-tuning. This \emph{reversibility} suggests that information is merely suppressed, not genuinely erased. To address this critical evaluation gap, we introduce a \emph{representation-level analysis framework}. Our toolkit comprises PCA similarity and shift, centered kernel alignment (CKA), and Fisher information, complemented by a summary metric, the mean PCA distance, to measure representational drift. Applying this framework across multiple unlearning methods, data domains, and LLMs, we identify four distinct forgetting regimes based on their \emph{reversibility} and \emph{catastrophicity}. We compare recovery strategies and show that relearning efficiency relies on the data source. We also find that irreversible, non-catastrophic forgetting is exceptionally challenging. By probing unlearning limits, we identify a case of seemingly irreversible, targeted forgetting, offering insights for more robust erasure algorithms. Overall, our findings expose a gap in current evaluation and establish a representation-level foundation for trustworthy unlearning.