Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Models

TL;DR

This paper investigates both overt and covert gender biases in machine translation models for Turkish and English, highlighting the need for nuanced, language-specific bias detection methods.

Contribution

It introduces a novel method to analyze asymmetrical gender markings and assesses bias in personhood attribution and stereotypes in MT models.

Findings

Covert biases are more deeply entrenched than overt biases.

Gender stereotypes are reflected in occupational and personality attributions.

Asymmetrical gender markings reveal hidden biases in translation models.

Abstract

As Machine Translation (MT) has become increasingly more powerful, accessible, and widespread, the potential for the perpetuation of bias has grown alongside its advances. While overt indicators of bias have been studied in machine translation, we argue that covert biases expose a problem that is further entrenched. Through the use of the gender-neutral language Turkish and the gendered language English, we examine cases of both overt and covert gender bias in MT models. Specifically, we introduce a method to investigate asymmetrical gender markings. We also assess bias in the attribution of personhood and examine occupational and personality stereotypes through overt bias indicators in MT models. Our work explores a deeper layer of bias in MT models and demonstrates the continued need for language-specific, interdisciplinary methodology in MT model development.