No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biases

TL;DR

This paper reveals that targeted bias mitigation in large language models often worsens biases in other categories and decreases overall coherence, highlighting the need for comprehensive evaluation methods.

Contribution

It provides a systematic analysis of cross-category effects of bias mitigation techniques across multiple models and biases, emphasizing the importance of multi-dimensional evaluation.

Findings

Targeted bias mitigation can increase biases in untargeted categories.

Mitigation often decreases model coherence and stereotypical preferences.

Robust evaluation tools are essential to prevent unintended bias exacerbation.

Abstract

Large Language Models (LLMs) inherit societal biases from their training data, potentially leading to harmful or unfair outputs. While various techniques aim to mitigate these biases, their effects are often evaluated only along the dimension of the bias being targeted. This work investigates the cross-category consequences of targeted bias mitigation. We study four bias mitigation techniques applied across ten models from seven model families, and we explore racial, religious, profession- and gender-related biases. We measure the impact of debiasing on model coherence and stereotypical preference using the StereoSet benchmark. Our results consistently show that while targeted mitigation can sometimes reduce bias in the intended dimension, it frequently leads to unintended and often negative consequences in others, such as increasing model bias and decreasing general coherence. These findings underscore the critical need for robust, multi-dimensional evaluation tools when examining and developing bias mitigation strategies to avoid inadvertently shifting or worsening bias along untargeted axes.