FairMonitor: A Four-Stage Automatic Framework for Detecting Stereotypes and Biases in Large Language Models

TL;DR

This paper presents FairMonitor, a comprehensive four-stage framework for detecting stereotypes and biases in LLMs' generated content, with a focus on interpretability and implicit bias detection, validated through an educational case study.

Contribution

Introduces a novel four-stage evaluation framework and automated metrics for bias detection in LLMs, addressing limitations of previous dataset-based methods.

Findings

Detected varying biases across five LLMs in education scenarios

Automated evaluation correlates highly with human annotations

Framework effectively identifies implicit stereotypes in generated content

Abstract

Detecting stereotypes and biases in Large Language Models (LLMs) can enhance fairness and reduce adverse impacts on individuals or groups when these LLMs are applied. However, the majority of existing methods focus on measuring the model's preference towards sentences containing biases and stereotypes within datasets, which lacks interpretability and cannot detect implicit biases and stereotypes in the real world. To address this gap, this paper introduces a four-stage framework to directly evaluate stereotypes and biases in the generated content of LLMs, including direct inquiry testing, serial or adapted story testing, implicit association testing, and unknown situation testing. Additionally, the paper proposes multi-dimensional evaluation metrics and explainable zero-shot prompts for automated evaluation. Using the education sector as a case study, we constructed the Edu-FairMonitor based on the four-stage framework, which encompasses 12,632 open-ended questions covering nine sensitive factors and 26 educational scenarios. Experimental results reveal varying degrees of stereotypes and biases in five LLMs evaluated on Edu-FairMonitor. Moreover, the results of our proposed automated evaluation method have shown a high correlation with human annotations.