Red-Teaming for Inducing Societal Bias in Large Language Models

TL;DR

This paper introduces two bias-specific red-teaming methods, EBP and BiasKG, to evaluate and induce societal bias in large language models, revealing that safety measures often fail to prevent bias.

Contribution

The paper presents novel bias-focused red-teaming strategies, EBP and BiasKG, specifically designed to evaluate and induce social bias in LLMs, highlighting gaps in current safety guardrails.

Findings

Bias increases in all models tested, including those with safety guardrails.

Bias induction methods effectively reveal societal biases in LLMs.

Evaluation underscores the need for improved bias mitigation in AI safety measures.

Abstract

Ensuring the safe deployment of AI systems is critical in industry settings where biased outputs can lead to significant operational, reputational, and regulatory risks. Thorough evaluation before deployment is essential to prevent these hazards. Red-teaming addresses this need by employing adversarial attacks to develop guardrails that detect and reject biased or harmful queries, enabling models to be retrained or steered away from harmful outputs. However, most red-teaming efforts focus on harmful or unethical instructions rather than addressing social bias, leaving this critical area under-explored despite its significant real-world impact, especially in customer-facing systems. We propose two bias-specific red-teaming methods, Emotional Bias Probe (EBP) and BiasKG, to evaluate how standard safety measures for harmful content affect bias. For BiasKG, we refactor natural language stereotypes into a knowledge graph. We use these attacking strategies to induce biased responses from several open- and closed-source language models. Unlike prior work, these methods specifically target social bias. We find our method increases bias in all models, even those trained with safety guardrails. Our work emphasizes uncovering societal bias in LLMs through rigorous evaluation, and recommends measures ensure AI safety in high-stakes industry deployments.