Differentially Private Estimation via Statistical Depth

TL;DR

This paper introduces new differentially private estimators for location and regression based on statistical depth functions, which are easier to analyze and typically have low influence, improving privacy guarantees especially in high-dimensional settings.

Contribution

It proposes novel DP estimators using statistical depth measures like halfspace and regression depth, including computationally efficient variants and methods satisfying random differential privacy.

Findings

Estimators perform well with sample sizes of 100-200 or higher privacy budgets.

Depth-based DP estimators have low influence and are easier to analyze.

Simulations show favorable performance compared to existing DP regression methods.

Abstract

Constructing a differentially private (DP) estimator requires deriving the maximum influence of an observation, which can be difficult in the absence of exogenous bounds on the input data or the estimator, especially in high dimensional settings. This paper shows that standard notions of statistical depth, i.e., halfspace depth and regression depth, are particularly advantageous in this regard, both in the sense that the maximum influence of a single observation is easy to analyze and that this value is typically low. This is used to motivate new approximate DP location and regression estimators using the maximizers of these two notions of statistical depth. A more computationally efficient variant of the approximate DP regression estimator is also provided. Also, to avoid requiring that users specify a priori bounds on the estimates and/or the observations, variants of these DP mechanisms are described that satisfy random differential privacy (RDP), which is a relaxation of differential privacy provided by Hall, Wasserman, and Rinaldo (2013). We also provide simulations of the two DP regression methods proposed here. The proposed estimators appear to perform favorably relative to the existing DP regression methods we consider in these simulations when either the sample size is at least 100-200 or the privacy-loss budget is sufficiently high.