Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruning

TL;DR

This paper introduces a movement pruning framework to analyze and reduce gender bias in pre-trained transformer models, revealing a bias-performance trade-off and proposing improvements to existing debiasing methods.

Contribution

We develop a novel movement pruning approach to identify less biased model subsets and enhance debiasing techniques in transformer-based language models.

Findings

Pruning attention heads can reduce gender bias.

A bias-performance trade-off exists in language models.

Our framework improves existing debiasing methods.

Abstract

Language model debiasing has emerged as an important field of study in the NLP community. Numerous debiasing techniques were proposed, but bias ablation remains an unaddressed issue. We demonstrate a novel framework for inspecting bias in pre-trained transformer-based language models via movement pruning. Given a model and a debiasing objective, our framework finds a subset of the model containing less bias than the original model. We implement our framework by pruning the model while fine-tuning it on the debiasing objective. Optimized are only the pruning scores - parameters coupled with the model's weights that act as gates. We experiment with pruning attention heads, an important building block of transformers: we prune square blocks, as well as establish a new way of pruning the entire heads. Lastly, we demonstrate the usage of our framework using gender bias, and based on our findings, we propose an improvement to an existing debiasing method. Additionally, we re-discover a bias-performance trade-off: the better the model performs, the more bias it contains.