Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?

TL;DR

This paper investigates how sharing components like training data or models among decision-makers can lead to outcome homogenization, potentially causing systemic exclusion and social hierarchy reinforcement, with empirical tests on fairness benchmarks.

Contribution

It formalizes the risk of outcome homogenization due to component sharing and empirically demonstrates its effects on fairness benchmarks and across different AI tasks.

Findings

Sharing training data increases outcome homogenization.

Individual effects of sharing are stronger than group effects.

Model adaptation methods influence outcome homogenization in foundation models.

Abstract

As the scope of machine learning broadens, we observe a recurring theme of algorithmic monoculture: the same systems, or systems that share components (e.g. training data), are deployed by multiple decision-makers. While sharing offers clear advantages (e.g. amortizing costs), does it bear risks? We introduce and formalize one such risk, outcome homogenization: the extent to which particular individuals or groups experience negative outcomes from all decision-makers. If the same individuals or groups exclusively experience undesirable outcomes, this may institutionalize systemic exclusion and reinscribe social hierarchy. To relate algorithmic monoculture and outcome homogenization, we propose the component-sharing hypothesis: if decision-makers share components like training data or specific models, then they will produce more homogeneous outcomes. We test this hypothesis on algorithmic fairness benchmarks, demonstrating that sharing training data reliably exacerbates homogenization, with individual-level effects generally exceeding group-level effects. Further, given the dominant paradigm in AI of foundation models, i.e. models that can be adapted for myriad downstream tasks, we test whether model sharing homogenizes outcomes across tasks. We observe mixed results: we find that for both vision and language settings, the specific methods for adapting a foundation model significantly influence the degree of outcome homogenization. We conclude with philosophical analyses of and societal challenges for outcome homogenization, with an eye towards implications for deployed machine learning systems.