Efficient Proposal-Test-Release for Minimax Optimal Estimation

TL;DR

This paper introduces an efficient Proposal-Test-Release (ePTR) method that simplifies differential privacy mechanisms, achieving near-optimal accuracy in various statistical estimation tasks by addressing sensitivity issues on atypical datasets.

Contribution

The paper develops ePTR, a simplified and computationally efficient variant of PTR, that maintains privacy guarantees while improving accuracy in minimax optimal estimation.

Findings

ePTR achieves rate-optimal accuracy in multiple estimation settings.

Empirical results show ePTR outperforms popular DP baselines.

Theoretically, ePTR provides privacy guarantees with simpler computations.

Abstract

Differential privacy (DP) is a rigorous framework that protects the participation of individuals in a dataset by limiting information leakage from released estimators. This creates a challenging setting for statisticians: DP must hold uniformly over all possible datasets, whereas statistical practice often downweights atypical or rare outcomes. The conceptual challenge is especially pronounced in sensitivity analysis, the key quantity governing the magnitude of DP noise and, consequently, estimator accuracy, because many estimators, including ordinary least squares for linear regression, exhibit markedly higher sensitivity on atypical datasets. Propose-Test-Release (PTR) is designed to address such cases, but its classical implementation requires computing the exact insensitive set and the dataset's Hellinger distance to that set, both of which are typically intractable. We introduce efficient PTR (ePTR), which replaces the exact insensitive set with a simpler subset and the exact Hellinger distance with a Lipschitz-based lower bound. This flexibility enables substantially simpler DP mechanisms that achieve rate-optimal accuracy in many settings. To illustrate, we study basic estimators for Bayes classification, linear regression, and nonparametric regression. We show that each can have high sensitivity on atypical datasets, yet admits intuitive ePTR-based designs. Empirically and theoretically, we compare ePTR against popular DP baselines and demonstrate improved accuracy while maintaining privacy guarantees.