Certifying Counterfactual Bias in LLMs

TL;DR

This paper introduces LLMCert-B, a novel framework that certifies large language models for counterfactual bias by providing high-confidence bounds on unbiased responses across diverse demographic prompts.

Contribution

It is the first framework to certify LLMs for counterfactual bias with guarantees over distributions of prompts, addressing scalability and reliability issues of prior bias evaluations.

Findings

Generated certificates for SOTA LLMs exposing vulnerabilities.

Applied certification to various prompt distributions including jailbreaks.

Demonstrated the framework's ability to identify biases in large models.

Abstract

Large Language Models (LLMs) can produce biased responses that can cause representational harms. However, conventional studies are insufficient to thoroughly evaluate biases across LLM responses for different demographic groups (a.k.a. counterfactual bias), as they do not scale to large number of inputs and do not provide guarantees. Therefore, we propose the first framework, LLMCert-B that certifies LLMs for counterfactual bias on distributions of prompts. A certificate consists of high-confidence bounds on the probability of unbiased LLM responses for any set of counterfactual prompts - prompts differing by demographic groups, sampled from a distribution. We illustrate counterfactual bias certification for distributions of counterfactual prompts created by applying prefixes sampled from prefix distributions, to a given set of prompts. We consider prefix distributions consisting random token sequences, mixtures of manual jailbreaks, and perturbations of jailbreaks in LLM's embedding space. We generate non-trivial certificates for SOTA LLMs, exposing their vulnerabilities over distributions of prompts generated from computationally inexpensive prefix distributions.