# Improved Differentially Private Analysis of Variance

## TL;DR

This paper introduces a new differentially private ANOVA test with higher statistical power, requiring significantly less data to detect effects compared to previous methods.

## Contribution

It develops a novel test statistic $F_1$ for private ANOVA, providing rigorous reference distribution computation and improved data efficiency.

## Key findings

- The $F_1$ statistic outperforms the traditional $F$-statistic in private settings.
- The new test achieves an order of magnitude better data efficiency.
- Experimental results show only 7% of the data needed compared to previous methods.

## Abstract

Hypothesis testing is one of the most common types of data analysis and forms the backbone of scientific research in many disciplines. Analysis of variance (ANOVA) in particular is used to detect dependence between a categorical and a numerical variable. Here we show how one can carry out this hypothesis test under the restrictions of differential privacy. We show that the $F$-statistic, the optimal test statistic in the public setting, is no longer optimal in the private setting, and we develop a new test statistic $F_1$ with much higher statistical power. We show how to rigorously compute a reference distribution for the $F_1$ statistic and give an algorithm that outputs accurate $p$-values. We implement our test and experimentally optimize several parameters. We then compare our test to the only previous work on private ANOVA testing, using the same effect size as that work. We see an order of magnitude improvement, with our test requiring only 7% as much data to detect the effect.