From Measurement to Mitigation: Exploring the Transferability of Debiasing Approaches to Gender Bias in Maltese Language Models

TL;DR

This paper evaluates gender bias in Maltese language models, adapting English bias mitigation techniques to a low-resource, morphologically rich language, and highlights the challenges and need for more inclusive NLP approaches.

Contribution

It adapts and assesses existing bias mitigation methods for Maltese LLMs, creating evaluation datasets and analyzing transferability challenges in low-resource languages.

Findings

Existing bias mitigation methods face challenges in Maltese due to linguistic complexity.

Adapting English bias techniques to Maltese requires significant modifications.

Highlighting the need for language-specific bias mitigation strategies.

Abstract

The advancement of Large Language Models (LLMs) has transformed Natural Language Processing (NLP), enabling performance across diverse tasks with little task-specific training. However, LLMs remain susceptible to social biases, particularly reflecting harmful stereotypes from training data, which can disproportionately affect marginalised communities. We measure gender bias in Maltese LMs, arguing that such bias is harmful as it reinforces societal stereotypes and fails to account for gender diversity, which is especially problematic in gendered, low-resource languages. While bias evaluation and mitigation efforts have progressed for English-centric models, research on low-resourced and morphologically rich languages remains limited. This research investigates the transferability of debiasing methods to Maltese language models, focusing on BERTu and mBERTu, BERT-based monolingual and multilingual models respectively. Bias measurement and mitigation techniques from English are adapted to Maltese, using benchmarks such as CrowS-Pairs and SEAT, alongside debiasing methods Counterfactual Data Augmentation, Dropout Regularization, Auto-Debias, and GuiDebias. We also contribute to future work in the study of gender bias in Maltese by creating evaluation datasets. Our findings highlight the challenges of applying existing bias mitigation methods to linguistically complex languages, underscoring the need for more inclusive approaches in the development of multilingual NLP.