Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs

TL;DR

This paper argues that in certain contexts, recognizing and measuring group differences in language models is crucial for fairness, introducing a benchmark suite to evaluate models' difference awareness and showing that existing mitigation strategies can sometimes be counterproductive.

Contribution

It introduces a new perspective on fairness emphasizing difference awareness, and provides a benchmark suite with 16,000 questions across eight scenarios to evaluate this dimension in language models.

Findings

Difference awareness is a distinct fairness dimension.

Existing bias mitigation can backfire when difference awareness is considered.

Benchmark results show variability in models' ability to recognize appropriate group differences.

Abstract

Algorithmic fairness has conventionally adopted the mathematically convenient perspective of racial color-blindness (i.e., difference unaware treatment). However, we contend that in a range of important settings, group difference awareness matters. For example, differentiating between groups may be necessary in legal contexts (e.g., the U.S. compulsory draft applies to men but not women) and harm assessments (e.g., referring to girls as ``terrorists'' may be less harmful than referring to Muslim people as such). Thus, in contrast to most fairness work, we study fairness through the perspective of treating people differently -- when it is contextually appropriate to. We first introduce an important distinction between descriptive (fact-based), normative (value-based), and correlation (association-based) benchmarks. This distinction is significant because each category requires separate interpretation and mitigation tailored to its specific characteristics. Then, we present a benchmark suite composed of eight different scenarios for a total of 16k questions that enables us to assess difference awareness. Finally, we show results across ten models that demonstrate difference awareness is a distinct dimension to fairness where existing bias mitigation strategies may backfire.