ForgetMark: Stealthy Fingerprint Embedding via Targeted Unlearning in Language Models

TL;DR

ForgetMark introduces a stealthy fingerprinting method for language models that uses targeted unlearning and probabilistic traces, avoiding fixed triggers and improving robustness and detectability over existing backdoor approaches.

Contribution

It proposes a novel fingerprinting framework that encodes provenance via targeted unlearning with lightweight adapters, enhancing stealthiness and robustness compared to prior backdoor methods.

Findings

Achieves 100% ownership verification across diverse models.

Maintains standard model performance while embedding fingerprints.

Outperforms backdoor baselines in stealthiness and robustness.

Abstract

Existing invasive (backdoor) fingerprints suffer from high-perplexity triggers that are easily filtered, fixed response patterns exposed by heuristic detectors, and spurious activations on benign inputs. We introduce \textsc{ForgetMark}, a stealthy fingerprinting framework that encodes provenance via targeted unlearning. It builds a compact, human-readable key--value set with an assistant model and predictive-entropy ranking, then trains lightweight LoRA adapters to suppress the original values on their keys while preserving general capabilities. Ownership is verified under black/gray-box access by aggregating likelihood and semantic evidence into a fingerprint success rate. By relying on probabilistic forgetting traces rather than fixed trigger--response patterns, \textsc{ForgetMark} avoids high-perplexity triggers, reduces detectability, and lowers false triggers. Across diverse architectures and settings, it achieves 100\% ownership verification on fingerprinted models while maintaining standard performance, surpasses backdoor baselines in stealthiness and robustness to model merging, and remains effective under moderate incremental fine-tuning. Our code and data are available at \href{https://github.com/Xuzhenhua55/ForgetMark}{https://github.com/Xuzhenhua55/ForgetMark}.