Social Bias Evaluation for Large Language Models Requires Prompt Variations

TL;DR

This study demonstrates that large language models are highly sensitive to prompt variations, which significantly affect their social bias and task performance, highlighting the need for diverse prompts in bias evaluation.

Contribution

The paper systematically analyzes how different prompt variations influence LLMs' social bias and performance, revealing prompt sensitivity and tradeoffs not thoroughly examined before.

Findings

LLMs' bias and performance fluctuate with prompt changes

Prompt diversity reduces bias but may lower task accuracy

Advanced LLMs exhibit output variability due to prompt ambiguity

Abstract

Warning: This paper contains examples of stereotypes and biases. Large Language Models (LLMs) exhibit considerable social biases, and various studies have tried to evaluate and mitigate these biases accurately. Previous studies use downstream tasks as prompts to examine the degree of social biases for evaluation and mitigation. While LLMs' output highly depends on prompts, previous studies evaluating and mitigating bias have often relied on a limited variety of prompts. In this paper, we investigate the sensitivity of LLMs when changing prompt variations (task instruction and prompt, few-shot examples, debias-prompt) by analyzing task performance and social bias of LLMs. Our experimental results reveal that LLMs are highly sensitive to prompts to the extent that the ranking of LLMs fluctuates when comparing models for task performance and social bias. Additionally, we show that LLMs have tradeoffs between performance and social bias caused by the prompts. Less bias from prompt setting may result in reduced performance. Moreover, the ambiguity of instances is one of the reasons for this sensitivity to prompts in advanced LLMs, leading to various outputs. We recommend using diverse prompts, as in this study, to compare the effects of prompts on social bias in LLMs.