On the Price of Privacy for Language Identification and Generation

TL;DR

This paper quantifies the fundamental privacy costs in language identification and generation tasks using differential privacy, showing that privacy has a surprisingly mild impact on error rates.

Contribution

It establishes tight bounds on the error rates for private language tasks, revealing minimal privacy costs under approximate DP and tight bounds under pure DP.

Findings

Approximate $(oldsymbol{ ext{}oldsymbol{ ext{(}}oldsymbol{ ext{}}oldsymbol{ ext{)}} ext{)}}$-DP recovers non-private error rates.

Pure $oldsymbol{ ext{DP}}$ causes a multiplicative degradation in error exponents.

Upper and lower bounds match under mild assumptions, showing optimal privacy-utility trade-offs.

Abstract

As large language models (LLMs) are increasingly trained on sensitive user data, understanding the fundamental cost of privacy in language learning becomes essential. We initiate the study of differentially private (DP) language identification and generation in the agnostic statistical setting, establishing algorithms and matching lower bounds that precisely quantify the cost of privacy. For both tasks, approximate $(\varepsilon, \delta)$-DP with constant $\varepsilon > 0$ recovers the non-private error rates: $\exp(-r(n))$ for identification (for any $r(n) = o(n)$) and $\exp(-\Omega(n))$ for generation. Under pure $\varepsilon$-DP, the exponents degrade by a multiplicative factor of $\min\{1, \varepsilon\}$, which we show is tight up to constants. Notably, for generation under pure DP with mild assumptions, the upper bound $\exp(-\min\{1,\varepsilon\} \cdot \Omega(n))$ matches the lower bound up to some constants, establishing an optimal rate. Our results show that the cost of privacy in language learning is surprisingly mild: absent entirely under approximate DP, and exactly a $\min\{1,\varepsilon\}$ factor in the exponent under pure DP.