Fairness in Socio-technical Systems: a Case Study of Wikipedia

TL;DR

This paper investigates fairness issues in Wikipedia as a socio-technical system, analyzing biases, their causes, and normative fairness expectations, highlighting the complexity beyond automated algorithms.

Contribution

First systematic review of fairness in a socio-technical system like Wikipedia, relating observed biases to harm and fairness criteria from algorithmic fairness research.

Findings

Biases in Wikipedia are complex and caused by socio-technical processes.

Observed harms relate to established notions of fairness and harm.

Existing fairness criteria may not fully apply to socio-technical contexts.

Abstract

Problems broadly known as algorithmic bias frequently occur in the context of complex socio-technical systems (STS), where observed biases may not be directly attributable to a single automated decision algorithm. As a first investigation of fairness in STS, we focus on the case of Wikipedia. We systematically review 75 papers describing different types of bias in Wikipedia, which we classify and relate to established notions of harm from algorithmic fairness research. By analysing causal relationships between the observed phenomena, we demonstrate the complexity of the socio-technical processes causing harm. Finally, we identify the normative expectations of fairness associated with the different problems and discuss the applicability of existing criteria proposed for machine learning-driven decision systems.