Intra-Fairness Dynamics: The Bias Spillover Effect in Targeted LLM Alignment

TL;DR

This paper explores how targeted fairness interventions in large language models can unintentionally cause bias spillover across multiple sensitive attributes, emphasizing the importance of context-aware fairness evaluation.

Contribution

It introduces the concept of bias spillover in LLM fairness, demonstrating its occurrence across nine attributes and highlighting the need for multi-attribute, context-sensitive fairness assessments.

Findings

Bias spillover occurs when fairness improvements in one attribute worsen others.

Context-aware analysis reveals significant bias degradation in ambiguous situations.

Targeted fairness interventions can inadvertently increase disparities across multiple attributes.

Abstract

Conventional large language model (LLM) fairness alignment largely focuses on mitigating bias along single sensitive attributes, overlooking fairness as an inherently multidimensional and context-specific value. This approach risks creating systems that achieve narrow fairness metrics while exacerbating disparities along untargeted attributes, a phenomenon known as bias spillover. While extensively studied in machine learning, bias spillover remains critically underexplored in LLM alignment. In this work, we investigate how targeted gender alignment affects fairness across nine sensitive attributes in three state-of-the-art LLMs (Mistral 7B, Llama 3.1 8B, Qwen 2.5 7B). Using Direct Preference Optimization and the BBQ benchmark, we evaluate fairness under ambiguous and disambiguous contexts. Our findings reveal noticeable bias spillover: while aggregate results show improvements, context-aware analysis exposes significant degradations in ambiguous contexts, particularly for physical appearance ($p< 0.001$ across all models), sexual orientation, and disability status. We demonstrate that improving fairness along one attribute can inadvertently worsen disparities in others under uncertainty, highlighting the necessity of context-aware, multi-attribute fairness evaluation frameworks.