Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts

TL;DR

This study evaluates social bias in Retrieval-Augmented Generation systems, revealing that external context can reduce stereotypes, but reasoning processes like Chain-of-Thought may increase bias, highlighting complex bias dynamics.

Contribution

It provides a comprehensive analysis of social bias in RAG systems, demonstrating bias reduction through external context and examining the impact of reasoning prompts on bias.

Findings

External context in RAG reduces bias in predictions.

Chain-of-Thought prompting can increase bias despite improving accuracy.

Bias shifts between stereotype and anti-stereotype responses with more context.

Abstract

Social biases inherent in large language models (LLMs) raise significant fairness concerns. Retrieval-Augmented Generation (RAG) architectures, which retrieve external knowledge sources to enhance the generative capabilities of LLMs, remain susceptible to the same bias-related challenges. This work focuses on evaluating and understanding the social bias implications of RAG. Through extensive experiments across various retrieval corpora, LLMs, and bias evaluation datasets, encompassing more than 13 different bias types, we surprisingly observe a reduction in bias in RAG. This suggests that the inclusion of external context can help counteract stereotype-driven predictions, potentially improving fairness by diversifying the contextual grounding of the model's outputs. To better understand this phenomenon, we then explore the model's reasoning process by integrating Chain-of-Thought (CoT) prompting into RAG while assessing the faithfulness of the model's CoT. Our experiments reveal that the model's bias inclinations shift between stereotype and anti-stereotype responses as more contextual information is incorporated from the retrieved documents. Interestingly, we find that while CoT enhances accuracy, contrary to the bias reduction observed with RAG, it increases overall bias across datasets, highlighting the need for bias-aware reasoning frameworks that can mitigate this trade-off.