AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings

TL;DR

This paper conducts a comprehensive analysis of biases in Arabic word embeddings across various models, dialects, and time periods, revealing how biases like gender stereotypes evolve and persist in different Arabic corpora.

Contribution

It introduces AraWEAT, a set of bias tests for Arabic embeddings, and provides an extensive multidimensional analysis of biases across models, dialects, and time.

Findings

Implicit gender bias increases over time in Arabic news embeddings

Bias varies across dialects and text types

Arabic embeddings encode significant societal biases

Abstract

Recent work has shown that distributional word vector spaces often encode human biases like sexism or racism. In this work, we conduct an extensive analysis of biases in Arabic word embeddings by applying a range of recently introduced bias tests on a variety of embedding spaces induced from corpora in Arabic. We measure the presence of biases across several dimensions, namely: embedding models (Skip-Gram, CBOW, and FastText) and vector sizes, types of text (encyclopedic text, and news vs. user-generated content), dialects (Egyptian Arabic vs. Modern Standard Arabic), and time (diachronic analyses over corpora from different time periods). Our analysis yields several interesting findings, e.g., that implicit gender bias in embeddings trained on Arabic news corpora steadily increases over time (between 2007 and 2017). We make the Arabic bias specifications (AraWEAT) publicly available.