A Feasibility Study of Differentially Private Summary Statistics and Regression Analyses with Evaluations on Administrative and Survey Data

TL;DR

This study evaluates the feasibility of differentially private methods for releasing summary statistics and regression analyses on sensitive administrative and survey data, highlighting their strengths and limitations for policy research.

Contribution

It introduces new adaptations of DP regression methods for complex data types and assesses their practical utility on real datasets, providing the first comprehensive analysis of this kind.

Findings

DP methods work well for simple statistics

DP regression estimates and confidence intervals are less accurate

Data complexity poses challenges for DP methods

Abstract

Federal administrative data, such as tax data, are invaluable for research, but because of privacy concerns, access to these data is typically limited to select agencies and a few individuals. An alternative to sharing microlevel data is to allow individuals to query statistics without directly accessing the confidential data. This paper studies the feasibility of using differentially private (DP) methods to make certain queries while preserving privacy. We also include new methodological adaptations to existing DP regression methods for using new data types and returning standard error estimates. We define feasibility as the impact of DP methods on analyses for making public policy decisions and the queries accuracy according to several utility metrics. We evaluate the methods using Internal Revenue Service data and public-use Current Population Survey data and identify how specific data features might challenge some of these methods. Our findings show that DP methods are feasible for simple, univariate statistics but struggle to produce accurate regression estimates and confidence intervals. To the best of our knowledge, this is the first comprehensive statistical study of DP regression methodology on real, complex datasets, and the findings have significant implications for the direction of a growing research field and public policy.