Richer Countries and Richer Representations

TL;DR

This paper investigates how country representation in language models is affected by frequency, revealing disparities linked to economic wealth and proposing strategies for fairer embeddings.

Contribution

It identifies the link between country frequency in training data and representational biases, and suggests reporting frequency and designing guarantees to mitigate disparities.

Findings

Countries with low frequency are less accurately represented.

Frequency correlates with GDP, perpetuating inequalities.

Mitigation strategies can improve representational fairness.

Abstract

We examine whether some countries are more richly represented in embedding space than others. We find that countries whose names occur with low frequency in training corpora are more likely to be tokenized into subwords, are less semantically distinct in embedding space, and are less likely to be correctly predicted: e.g., Ghana (the correct answer and in-vocabulary) is not predicted for, "The country producing the most cocoa is [MASK].". Although these performance discrepancies and representational harms are due to frequency, we find that frequency is highly correlated with a country's GDP; thus perpetuating historic power and wealth inequalities. We analyze the effectiveness of mitigation strategies; recommend that researchers report training word frequencies; and recommend future work for the community to define and design representational guarantees.