Auditing for Spatial Fairness

TL;DR

This paper introduces a statistical framework for assessing spatial fairness in algorithms by testing whether outcomes are independent of location, addressing limitations of existing discretization methods and gerrymandering concerns.

Contribution

It proposes a likelihood ratio test-based method to evaluate spatial fairness, allowing for localized discrepancies and improving over prior approaches that only handle regular location grids.

Findings

The proposed test effectively detects spatial unfairness in algorithms.

It accounts for localized outcome disparities without relying on fixed location groups.

The method provides a statistically rigorous way to assess spatial fairness.

Abstract

This paper studies algorithmic fairness when the protected attribute is location. To handle protected attributes that are continuous, such as age or income, the standard approach is to discretize the domain into predefined groups, and compare algorithmic outcomes across groups. However, applying this idea to location raises concerns of gerrymandering and may introduce statistical bias. Prior work addresses these concerns but only for regularly spaced locations, while raising other issues, most notably its inability to discern regions that are likely to exhibit spatial unfairness. Similar to established notions of algorithmic fairness, we define spatial fairness as the statistical independence of outcomes from location. This translates into requiring that for each region of space, the distribution of outcomes is identical inside and outside the region. To allow for localized discrepancies in the distribution of outcomes, we compare how well two competing hypotheses explain the observed outcomes. The null hypothesis assumes spatial fairness, while the alternate allows different distributions inside and outside regions. Their goodness of fit is then assessed by a likelihood ratio test. If there is no significant difference in how well the two hypotheses explain the observed outcomes, we conclude that the algorithm is spatially fair.