Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning

TL;DR

This paper evaluates honesty in large language model unlearning, introduces metrics for assessment, finds current methods lacking, and proposes ReVa to improve honesty and forgetting effectiveness.

Contribution

It defines unlearning honesty, develops evaluation metrics, and introduces ReVa, a method that enhances honesty and forgetting in LLM unlearning.

Findings

Current unlearning methods often hallucinate and behave inconsistently.

ReVa nearly doubles rejection rate and improves honesty on retained knowledge.

All evaluated methods fail to fully meet honesty and forgetting standards.

Abstract

Unlearning in large language models (LLMs) aims to remove harmful training data while preserving overall utility. However, we find that existing methods often hallucinate, generate abnormal token sequences, or behave inconsistently, raising safety and trust concerns. According to prior literature on LLM honesty, such behaviors are often associated with dishonesty. This motivates us to investigate the notion of honesty in the context of model unlearning. We propose a formal definition of unlearning honesty, which includes: (1) preserving both utility and honesty on retained knowledge, and (2) ensuring effective forgetting while encouraging the model to acknowledge its limitations and respond consistently to questions related to forgotten knowledge. To systematically evaluate the honesty of unlearning, we introduce a suite of metrics that cover utility, honesty on the retained set, effectiveness of forgetting, rejection rate and refusal stability in Q&A and MCQ settings. Evaluating 9 methods across 3 mainstream families shows that all current methods fail to meet these standards. After experimental and theoretical analyses, we present ReVa, a representation-alignment procedure that fine-tunes feature-randomized unlearned models to better acknowledge forgotten knowledge. On Q&A tasks from the forget set, ReVa achieves the highest rejection rate after two rounds of interaction, nearly doubling the performance of the second-best method. Remarkably, It also improves honesty on the retained set. We release our data and code at https://github.com/renjiegu.