Generalizing Fairness to Generative Language Models via Reformulation of Non-discrimination Criteria

TL;DR

This paper develops methods to identify and measure gender bias in large language models by adapting fairness criteria from classification, focusing on occupational stereotypes in medical contexts.

Contribution

It introduces generative AI analogues of non-discrimination fairness criteria and applies them to detect occupational gender bias in language models.

Findings

Identified gender bias in medical occupational stereotypes.

Demonstrated applicability of fairness criteria to generative models.

Provided prompts to measure bias in conversational AI.

Abstract

Generative AI, such as large language models, has undergone rapid development within recent years. As these models become increasingly available to the public, concerns arise about perpetuating and amplifying harmful biases in applications. Gender stereotypes can be harmful and limiting for the individuals they target, whether they consist of misrepresentation or discrimination. Recognizing gender bias as a pervasive societal construct, this paper studies how to uncover and quantify the presence of gender biases in generative language models. In particular, we derive generative AI analogues of three well-known non-discrimination criteria from classification, namely independence, separation and sufficiency. To demonstrate these criteria in action, we design prompts for each of the criteria with a focus on occupational gender stereotype, specifically utilizing the medical test to introduce the ground truth in the generative AI context. Our results address the presence of occupational gender bias within such conversational language models.