Comparing Biases and the Impact of Multilingual Training across Multiple Languages

TL;DR

This paper analyzes social biases in sentiment analysis across five languages, comparing biases in monolingual and multilingual models, revealing cultural favoritism and bias amplification effects.

Contribution

It introduces a multilingual bias analysis framework across five languages and examines how multilingual training influences bias expression and variation.

Findings

Biases reflect cultural dominance in each language.

Multilingual finetuning increases bias variation and amplification.

Bias patterns differ across languages and attributes.

Abstract

Studies in bias and fairness in natural language processing have primarily examined social biases within a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across various languages for individual attributes. As a result, it is critical to examine biases within each language and attribute. Of equal importance is to study how these biases compare across languages and how the biases are affected when training a model on multilingual data versus monolingual data. We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task to observe whether specific demographics are viewed more positively. We study bias similarities and differences across these languages and investigate the impact of multilingual vs. monolingual training data. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture (e.g. majority religions and nationalities). Additionally, we find an increased variation in predictions across protected groups, indicating bias amplification, after multilingual finetuning in comparison to multilingual pretraining.