Context matters for fairness -- a case study on the effect of spatial distribution shifts

TL;DR

This paper investigates how spatial distribution shifts impact the fairness and performance of AI models, emphasizing the importance of context and robustness when deploying models across different regions.

Contribution

It provides a case study demonstrating the effects of spatial distribution shifts on fairness and model performance, highlighting the need for robustness in fairness-aware models.

Findings

Spatial distribution shifts significantly affect model fairness and accuracy.

Fairness interventions vary in effectiveness across different states and groups.

Robustness to distribution shifts is crucial for fair AI deployment.

Abstract

With the ever growing involvement of data-driven AI-based decision making technologies in our daily social lives, the fairness of these systems is becoming a crucial phenomenon. However, an important and often challenging aspect in utilizing such systems is to distinguish validity for the range of their application especially under distribution shifts, i.e., when a model is deployed on data with different distribution than the training set. In this paper, we present a case study on the newly released American Census datasets, a reconstruction of the popular Adult dataset, to illustrate the importance of context for fairness and show how remarkably can spatial distribution shifts affect predictive- and fairness-related performance of a model. The problem persists for fairness-aware learning models with the effects of context-specific fairness interventions differing across the states and different population groups. Our study suggests that robustness to distribution shifts is necessary before deploying a model to another context.