The Privacy Price of Tail-Risk Learning: Effective Tail Sample Size in Differentially Private CVaR Optimization

TL;DR

This paper analyzes how differential privacy impacts the effective sample size in CVaR learning, revealing a privacy cost that scales with tail mass, privacy parameters, and problem complexity.

Contribution

It provides complete theoretical rates for private CVaR estimation, decomposing the privacy impact and establishing the effective tail sample size as the core challenge.

Findings

Effective private tail sample size is $ ext{epsilon} imes n imes au$.

Rates for scalar estimation and finite classes are characterized explicitly.

Privacy cost in convex Lipschitz learning scales as $1/( ext{epsilon} imes n imes au)$.

Abstract

Differential privacy changes the effective sample size governing CVaR learning. For tail mass $\tau$, the privacy-relevant sample size is not $n$, but $n\tau$; equivalently, the effective private tail sample size is $\epsilon n\tau$. Private CVaR excess risk decomposes into ordinary tail-risk statistical error and a privacy price. This decomposition is complete for scalar estimation and finite classes: scalar estimation has rate $\Theta(B \min\{1,(n\tau)^{-1/2}+(\epsilon n\tau)^{-1}\})$, and finite classes of size $M$ have rate $\Theta(B \min\{1,\sqrt{\log(2M)/(n\tau)}+\log(2M)/(\epsilon n\tau)\})$. These complete rates hold under pure DP, and their lower bounds extend to approximate DP in the stated small-$\delta$ regimes. For convex Lipschitz learning, modular upper and lower reductions show that the CVaR-specific privacy term necessarily scales as $1/(\epsilon n\tau)$, with dimension dependence inherited from private stochastic convex optimization. Together, these results identify ordinary private learning on $\Theta(n\tau)$ informative tail records as the canonical hard subproblem inside private CVaR learning.