Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translation

TL;DR

This paper introduces 2DAMA, a dual debiasing algorithm that reduces gender stereotypes in language models and translation while preserving factual gender information, enhancing fairness without sacrificing model capabilities.

Contribution

The paper presents a novel dual debiasing method that effectively mitigates stereotypes and maintains factual gender cues in language models and translation tasks.

Findings

Reduces gender bias in English language models

Mitigates stereotypical tendencies in translation

Preserves factual gender information

Abstract

Mitigation of biases, such as language models' reliance on gender stereotypes, is a crucial endeavor required for the creation of reliable and useful language technology. The crucial aspect of debiasing is to ensure that the models preserve their versatile capabilities, including their ability to solve language tasks and equitably represent various genders. To address this issue, we introduce a streamlined Dual Dabiasing Algorithm through Model Adaptation (2DAMA). Novel Dual Debiasing enables robust reduction of stereotypical bias while preserving desired factual gender information encoded by language models. We show that 2DAMA effectively reduces gender bias in English and is one of the first approaches facilitating the mitigation of stereotypical tendencies in translation. The proposed method's key advantage is the preservation of factual gender cues, which are useful in a wide range of natural language processing tasks.