Revisiting Differentially Private Hypothesis Tests for Categorical Data

TL;DR

This paper develops new differentially private hypothesis tests for categorical data that improve power and reliability by adjusting for privacy-induced noise, applicable to various statistical tests.

Contribution

It introduces practical, bias-corrected differentially private tests for categorical data, using a new asymptotic regime and modified test equivalences.

Findings

Enhanced test power under differential privacy

Reliable p-values with bias correction

Effective on diverse datasets and privacy levels

Abstract

In this paper, we consider methods for performing hypothesis tests on data protected by a statistical disclosure control technology known as differential privacy. Previous approaches to differentially private hypothesis testing either perturbed the test statistic with random noise having large variance (and resulted in a significant loss of power) or added smaller amounts of noise directly to the data but failed to adjust the test in response to the added noise (resulting in biased, unreliable $p$-values). In this paper, we develop a variety of practical hypothesis tests that address these problems. Using a different asymptotic regime that is more suited to hypothesis testing with privacy, we show a modified equivalence between chi-squared tests and likelihood ratio tests. We then develop differentially private likelihood ratio and chi-squared tests for a variety of applications on tabular data (i.e., independence, sample proportions, and goodness-of-fit tests). Experimental evaluations on small and large datasets using a wide variety of privacy settings demonstrate the practicality and reliability of our methods.