Provably Protecting Fine-Tuned LLMs from Training Data Extraction while Preserving Utility

TL;DR

This paper introduces SCP-Δr, a novel method that enhances privacy of fine-tuned LLMs against data extraction attacks while maintaining high utility, through smoothing low-impact tokens based on probability shifts.

Contribution

It proposes SCP-Δr, a new NAF-based algorithm that offers stronger theoretical privacy guarantees and empirical protection with minimal utility degradation.

Findings

SCP-Δr significantly reduces data extraction risks.

The method maintains model utility with minimal performance loss.

It outperforms existing approaches in theoretical privacy bounds.

Abstract

Fine-tuning large language models (LLMs) on sensitive datasets raises privacy concerns, as training data extraction (TDE) attacks can expose highly confidential information. Existing defenses against such attacks either lack formal privacy guarantees or incur substantial utility degradation. We observe that fine-tuning induces widespread probability shifts, yet preserving only a small subset of influential token-level deviations is sufficient; the remaining shifts can be aggressively smoothed with minimal impact on utility. Motivated by this insight, we propose SCP-$\Delta_r$, a Near Access Freeness (NAF)-based algorithm that operates on relative probabilities and explicitly smooths low-impact tokens using a base model. SCP-$\Delta_r$ achieves orders-of-magnitude better theoretical bounds than existing NAF based methods and provides strong empirical protection against TDE attacks with minimal performance loss.