Social Biases in Knowledge Representations of Wikidata separates Global North from Global South

TL;DR

This paper introduces AuditLP, a framework for detecting social biases in knowledge graph link prediction, revealing geographic disparities that reflect global socio-economic and cultural divisions.

Contribution

The paper presents a novel fairness assessment framework for knowledge graph link prediction, highlighting geographic bias patterns in Wikidata.

Findings

Occupations are classified as male or female-dominated based on gender bias.

Age-related biases show occupations as young-biased or old-biased.

Bias outcomes vary across geographies, reflecting global socio-economic divisions.

Abstract

Knowledge Graphs have become increasingly popular due to their wide usage in various downstream applications, including information retrieval, chatbot development, language model construction, and many others. Link prediction (LP) is a crucial downstream task for knowledge graphs, as it helps to address the problem of the incompleteness of the knowledge graphs. However, previous research has shown that knowledge graphs, often created in a (semi) automatic manner, are not free from social biases. These biases can have harmful effects on downstream applications, especially by leading to unfair behavior toward minority groups. To understand this issue in detail, we develop a framework -- AuditLP -- deploying fairness metrics to identify biased outcomes in LP, specifically how occupations are classified as either male or female-dominated based on gender as a sensitive attribute. We have experimented with the sensitive attribute of age and observed that occupations are categorized as young-biased, old-biased, and age-neutral. We conduct our experiments on a large number of knowledge triples that belong to 21 different geographies extracted from the open-sourced knowledge graph, Wikidata. Our study shows that the variance in the biased outcomes across geographies neatly mirrors the socio-economic and cultural division of the world, resulting in a transparent partition of the Global North from the Global South.