PerProb: Indirectly Evaluating Memorization in Large Language Models

TL;DR

PerProb introduces a standardized, label-free framework for indirectly assessing memorization and privacy risks in large language models by analyzing perplexity differences, applicable across various models and settings.

Contribution

We propose PerProb, a novel, unified method for evaluating LLM memorization without relying on labels or internal model access, addressing limitations of prior MIAs.

Findings

PerProb effectively detects memorization across multiple datasets.

Mitigation strategies like differential privacy reduce data leakage.

Memory behaviors vary significantly among different LLMs.

Abstract

The rapid advancement of Large Language Models (LLMs) has been driven by extensive datasets that may contain sensitive information, raising serious privacy concerns. One notable threat is the Membership Inference Attack (MIA), where adversaries infer whether a specific sample was used in model training. However, the true impact of MIA on LLMs remains unclear due to inconsistent findings and the lack of standardized evaluation methods, further complicated by the undisclosed nature of many LLM training sets. To address these limitations, we propose PerProb, a unified, label-free framework for indirectly assessing LLM memorization vulnerabilities. PerProb evaluates changes in perplexity and average log probability between data generated by victim and adversary models, enabling an indirect estimation of training-induced memory. Compared with prior MIA methods that rely on member/non-member labels or internal access, PerProb is independent of model and task, and applicable in both black-box and white-box settings. Through a systematic classification of MIA into four attack patterns, we evaluate PerProb's effectiveness across five datasets, revealing varying memory behaviors and privacy risks among LLMs. Additionally, we assess mitigation strategies, including knowledge distillation, early stopping, and differential privacy, demonstrating their effectiveness in reducing data leakage. Our findings offer a practical and generalizable framework for evaluating and improving LLM privacy.