The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated

TL;DR

This paper investigates how current benchmarks may underestimate the true impact of debiasing methods on language models, showing that analyzing bias-specific instances provides more accurate evaluations.

Contribution

It demonstrates that debiasing effects are consistently underestimated in standard benchmarks and proposes analyzing bias-related instances separately for better assessment.

Findings

Debiasing effects are underestimated across multiple tasks.

Analyzing bias-specific instances yields more reliable evaluation.

Benchmark datasets may not fully capture debiasing impacts.

Abstract

Pre-trained language models trained on large-scale data have learned serious levels of social biases. Consequently, various methods have been proposed to debias pre-trained models. Debiasing methods need to mitigate only discriminatory bias information from the pre-trained models, while retaining information that is useful for the downstream tasks. In previous research, whether useful information is retained has been confirmed by the performance of downstream tasks in debiased pre-trained models. On the other hand, it is not clear whether these benchmarks consist of data pertaining to social biases and are appropriate for investigating the impact of debiasing. For example in gender-related social biases, data containing female words (e.g. ``she, female, woman''), male words (e.g. ``he, male, man''), and stereotypical words (e.g. ``nurse, doctor, professor'') are considered to be the most affected by debiasing. If there is not much data containing these words in a benchmark dataset for a target task, there is the possibility of erroneously evaluating the effects of debiasing. In this study, we compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets that containing female, male, and stereotypical words. Experiments show that the effects of debiasing are consistently \emph{underestimated} across all tasks. Moreover, the effects of debiasing could be reliably evaluated by separately considering instances containing female, male, and stereotypical words than all of the instances in a benchmark dataset.