Convex Optimization with Local Label Differential Privacy: Tight Bounds in All Privacy Regimes

TL;DR

This paper introduces a new non-interactive algorithm for stochastic convex optimization under local label differential privacy, achieving tighter bounds and reducing label space dependence from linear to square root.

Contribution

It presents the first efficient non-interactive L-LDP algorithm with improved risk bounds and proves a matching lower bound, resolving the fundamental cost question.

Findings

Achieves excess risk of O(√(K/εn)) in high-privacy regime

Achieves excess risk of O(√(K/e^ε n)) in medium-privacy regime

Quadratically improves label space dependence from linear to square root

Abstract

We study the problem of Stochastic Convex Optimization (SCO) under the constraint of local Label Differential Privacy (L-LDP). In this setting, the features are considered public, but the corresponding labels are sensitive and must be randomized by each user locally before being sent to an untrusted analyzer. Prior work for SCO under L-LDP (Ghazi et al., 2021) established an excess population risk bound with a \emph{linear} dependence on the size of the label space, $K$: $O\left({\frac{K}{\epsilon\sqrt{n}}}\right)$ in the high-privacy regime ($\epsilon \leq 1$) and $O\left({\frac{K}{e^{\epsilon} \sqrt{n}}}\right)$ in the medium-privacy regime ($1 \leq \epsilon \leq \ln K$). This left open whether this linear cost is fundamental to the L-LDP model. In this note, we resolve this question. First, we present a novel and efficient non-interactive L-LDP algorithm that achieves an excess risk of $O\left({\sqrt{\frac{K}{\epsilon n}}}\right)$ in the high-privacy regime ($\epsilon \leq 1$) and $O\left({\sqrt{\frac{K}{e^{\epsilon} n}}}\right)$ in the medium-privacy regime ($1 \leq \epsilon \leq \ln K$). This quadratically improves the dependency on the label space size from $O(K)$ to $O(\sqrt{K})$. Second, we prove a matching information-theoretic lower bound across all privacy regimes for any sufficiently large $n$.