Probability of Differentiation Reveals Brittleness of Homogeneity Bias in GPT-4

TL;DR

This paper investigates the homogeneity bias in GPT-4 by analyzing the variability of its outputs across different prompts, revealing that such bias is highly volatile and sensitive to minor prompt changes, challenging previous assumptions based on encoder models.

Contribution

The study introduces a novel method to measure homogeneity bias in LLMs directly from their outputs, bypassing encoder models, and demonstrates that this bias is more volatile and brittle than previously thought.

Findings

Homogeneity bias varies significantly across prompts and cues.

Bias expression in GPT-4 is highly sensitive to minor prompt changes.

Previous encoder-based studies may have overestimated bias stability.

Abstract

Homogeneity bias in Large Language Models (LLMs) refers to their tendency to homogenize the representations of some groups compared to others. Previous studies documenting this bias have predominantly used encoder models, which may have inadvertently introduced biases. To address this limitation, we prompted GPT-4 to generate single word/expression completions associated with 18 situation cues-specific, measurable elements of environments that influence how individuals perceive situations and compared the variability of these completions using probability of differentiation. This approach directly assessed homogeneity bias from the model's outputs, bypassing encoder models. Across five studies, we find that homogeneity bias is highly volatile across situation cues and writing prompts, suggesting that the bias observed in past work may reflect those within encoder models rather than LLMs. Furthermore, we find that homogeneity bias in LLMs is brittle, as even minor and arbitrary changes in prompts can significantly alter the expression of biases. Future work should further explore how variations in syntactic features and topic choices in longer text generations influence homogeneity bias in LLMs.