Harry Potter is Still Here! Probing Knowledge Leakage in Targeted Unlearned Large Language Models via Automated Adversarial Prompting

TL;DR

This paper introduces LURK, a framework that uses adversarial prompts to detect hidden knowledge in unlearned large language models, revealing limitations in current unlearning evaluation methods.

Contribution

LURK provides a novel automated adversarial probing method to uncover residual knowledge in unlearned models, improving assessment of unlearning effectiveness.

Findings

Models believed to be unlearned still leak knowledge under targeted prompts.

Current unlearning standards may overestimate model unlearning success.

LURK offers a more rigorous evaluation of unlearning robustness.

Abstract

This work presents LURK (Latent UnleaRned Knowledge), a novel framework that probes for hidden retained knowledge in unlearned LLMs through adversarial suffix prompting. LURK automatically generates adversarial prompt suffixes designed to elicit residual knowledge about the Harry Potter domain, a commonly used benchmark for unlearning. Our experiments reveal that even models deemed successfully unlearned can leak idiosyncratic information under targeted adversarial conditions, highlighting critical limitations of current unlearning evaluation standards. By uncovering latent knowledge through indirect probing, LURK offers a more rigorous and diagnostic tool for assessing the robustness of unlearning algorithms. All code will be publicly available.