Unbiased Statistical Estimation and Valid Confidence Intervals Under Differential Privacy

TL;DR

This paper introduces a novel method for obtaining unbiased statistical estimates and valid confidence intervals under differential privacy constraints, applicable to high-dimensional data and general estimation procedures.

Contribution

It develops a general approach using the bag of little bootstraps and a generalized CoinPress algorithm to produce unbiased estimates and confidence intervals under differential privacy.

Findings

Produces unbiased estimates with high probability

Applicable to high-dimensional data and various estimators

Valid confidence intervals under privacy constraints

Abstract

We present a method for producing unbiased parameter estimates and valid confidence intervals under the constraints of differential privacy, a formal framework for limiting individual information leakage from sensitive data. Prior work in this area is limited in that it is tailored to calculating confidence intervals for specific statistical procedures, such as mean estimation or simple linear regression. While other recent work can produce confidence intervals for more general sets of procedures, they either yield only approximately unbiased estimates, are designed for one-dimensional outputs, or assume significant user knowledge about the data-generating distribution. Our method induces distributions of mean and covariance estimates via the bag of little bootstraps (BLB) and uses them to privately estimate the parameters' sampling distribution via a generalized version of the CoinPress estimation algorithm. If the user can bound the parameters of the BLB-induced parameters and provide heavier-tailed families, the algorithm produces unbiased parameter estimates and valid confidence intervals which hold with arbitrarily high probability. These results hold in high dimensions and for any estimation procedure which behaves nicely under the bootstrap.