Reducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case Study

TL;DR

This study investigates whether economic inequality metrics like the Gini coefficient can serve as proxies for demographic disparities in social media engagement, potentially aiding fairness assessments when demographic data is unavailable.

Contribution

It empirically demonstrates the positive correlation between demographic-free inequality metrics and demographic bias metrics on Twitter, suggesting their utility as proxy measures.

Findings

Gini coefficient correlates with demographic disparities in engagement.

Inequality metrics can serve as proxies for demographic bias.

Results support using economic inequality measures to assess fairness.

Abstract

Many existing fairness metrics measure group-wise demographic disparities in system behavior or model performance. Calculating these metrics requires access to demographic information, which, in industrial settings, is often unavailable. By contrast, economic inequality metrics, such as the Gini coefficient, require no demographic data to measure. However, reductions in economic inequality do not necessarily correspond to reductions in demographic disparities. In this paper, we empirically explore the relationship between demographic-free inequality metrics -- such as the Gini coefficient -- and standard demographic bias metrics that measure group-wise model performance disparities specifically in the case of engagement inequality on Twitter. We analyze tweets from 174K users over the duration of 2021 and find that demographic-free impression inequality metrics are positively correlated with gender, race, and age disparities in the average case, and weakly (but still positively) correlated with demographic bias in the worst case. We therefore recommend inequality metrics as a potentially useful proxy measure of average group-wise disparities, especially in cases where such disparities cannot be measured directly. Based on these results, we believe they can be used as part of broader efforts to improve fairness between demographic groups in scenarios like content recommendation on social media.