Geometrizing rates of convergence under local differential privacy constraints

TL;DR

This paper investigates the impact of local differential privacy constraints on the rates of convergence in functional estimation, providing theoretical bounds, optimal mechanisms, and illustrating the privacy-accuracy trade-offs across various problems.

Contribution

It introduces a new theoretical framework linking privacy constraints to convergence rates using the modulus of continuity, and offers practical guidelines for optimal privatization mechanisms.

Findings

Minimax risk is characterized by the modulus of continuity under privacy constraints.

A simple sample mean estimator achieves optimal rates in linear functional estimation.

The privacy cost varies significantly across different estimation problems.

Abstract

We study the problem of estimating a functional $\theta(\mathbb P)$ of an unknown probability distribution $\mathbb P \in\mathcal P$ in which the original iid sample $X_1,\dots, X_n$ is kept private even from the statistician via an $\alpha$-local differential privacy constraint. Let $\omega_{TV}$ denote the modulus of continuity of the functional $\theta$ over $\mathcal P$, with respect to total variation distance. For a large class of loss functions $l$ and a fixed privacy level $\alpha$, we prove that the privatized minimax risk is equivalent to $l(\omega_{TV}(n^{-1/2}))$ to within constants, under regularity conditions that are satisfied, in particular, if $\theta$ is linear and $\mathcal P$ is convex. Our results complement the theory developed by Donoho and Liu (1991) with the nowadays highly relevant case of privatized data. Somewhat surprisingly, the difficulty of the estimation problem in the private case is characterized by $\omega_{TV}$, whereas, it is characterized by the Hellinger modulus of continuity if the original data $X_1,\dots, X_n$ are available. We also find that for locally private estimation of linear functionals over a convex model a simple sample mean estimator, based on independently and binary privatized observations, always achieves the minimax rate. We further provide a general recipe for choosing the functional parameter in the optimal binary privatization mechanisms and illustrate the general theory in numerous examples. Our theory allows to quantify the price to be paid for local differential privacy in a large class of estimation problems. This price appears to be highly problem specific.