Instance-Optimal Private Density Estimation in the Wasserstein Distance

TL;DR

This paper develops and analyzes algorithms for differentially private density estimation in Wasserstein distance, achieving instance-optimal rates that adapt to the complexity of the underlying distribution.

Contribution

It introduces instance-optimal algorithms for private density estimation in Wasserstein distance over real line and two-dimensional spaces, extending to general metric spaces.

Findings

Achieves instance-optimal rates in 1D and 2D Wasserstein density estimation.

Extends to arbitrary metric spaces using hierarchical tree structures.

Provides instance-optimal private learning results for discrete distributions in TV distance.

Abstract

Estimating the density of a distribution from samples is a fundamental problem in statistics. In many practical settings, the Wasserstein distance is an appropriate error metric for density estimation. For example, when estimating population densities in a geographic region, a small Wasserstein distance means that the estimate is able to capture roughly where the population mass is. In this work we study differentially private density estimation in the Wasserstein distance. We design and analyze instance-optimal algorithms for this problem that can adapt to easy instances. For distributions $P$ over $\mathbb{R}$, we consider a strong notion of instance-optimality: an algorithm that uniformly achieves the instance-optimal estimation rate is competitive with an algorithm that is told that the distribution is either $P$ or $Q_P$ for some distribution $Q_P$ whose probability density function (pdf) is within a factor of 2 of the pdf of $P$. For distributions over $\mathbb{R}^2$, we use a different notion of instance optimality. We say that an algorithm is instance-optimal if it is competitive with an algorithm that is given a constant-factor multiplicative approximation of the density of the distribution. We characterize the instance-optimal estimation rates in both these settings and show that they are uniformly achievable (up to polylogarithmic factors). Our approach for $\mathbb{R}^2$ extends to arbitrary metric spaces as it goes via hierarchically separated trees. As a special case our results lead to instance-optimal private learning in TV distance for discrete distributions.