Private Hypothesis Testing for Social Sciences

TL;DR

This paper examines how differential privacy affects statistical power in social science experiments, analyzing sample size requirements and proposing empirical methods to enhance private statistical accuracy.

Contribution

It provides a theoretical analysis of sample size adjustments under differential privacy and introduces an empirical bootstrapping approach to improve private statistic accuracy.

Findings

Differential privacy increases required sample sizes for high power.

Gaussian mechanisms impact sample size calculations.

Bootstrapping improves accuracy of private statistics.

Abstract

While running any experiment, we often have to consider the statistical power to ensure an effective study. Statistical power or power ensures that we can observe an effect with high probability if such a true effect exists. However, several studies lack the appropriate planning for determining the optimal sample size to ensure adequate power. Thus, careful planning ensures that the power remains high even under high measurement errors while keeping the type 1 error constrained. We study the impact of differential privacy on experiments and theoretically analyze the change in sample size required due to the Gaussian mechanisms. Further, we provide an empirical method to improve the accuracy of private statistics with simple bootstrapping.