Empirical Power Analysis of a Statistical Test to Quantify Gerrymandering

TL;DR

This paper empirically evaluates the effectiveness of a statistical outlier test in detecting gerrymandering, revealing its stability across various conditions and highlighting the impact of bias metrics on its power.

Contribution

It provides the first computational verification of the power of statistical tests used in gerrymandering legal cases, focusing on the influence of bias metrics.

Findings

The test's power is stable across parties, election years, and chain lengths.

Choice of bias metric significantly affects test power.

The study highlights conditions under which the test may fail to detect gerrymandering.

Abstract

Gerrymandering is a pervasive problem within the US political system. In the past decade, methods based on Markov Chain Monte Carlo (MCMC) sampling and statistical outlier tests have been proposed to quantify gerrymandering and were used as evidence in several high-profile legal cases. We perform an empirical power analysis of one such hypothesis test from Chikina et al (2020). We generate a family of biased North Carolina congressional district maps using the 2012 and 2016 presidential elections and assess under which conditions the outlier test fails to flag them at the specified Type I error level. The power of the outlier test is found to be relatively stable across political parties, election years, lengths of the MCMC chain and effect sizes. The main effect on the power of the test is shown to be the choice of the bias metric. This is the first work that computationally verifies the power of statistical tests used in gerrymandering cases.