LLMs are Biased Evaluators But Not Biased for Retrieval Augmented Generation

TL;DR

This study investigates whether large language models exhibit bias in retrieval-augmented generation tasks, finding they are not biased towards self-generated content but are influenced by factual accuracy across multiple datasets and models.

Contribution

The paper demonstrates that LLMs do not show self-preference bias in RAG frameworks and highlights the importance of factual accuracy in their evaluation, contrasting prior bias findings.

Findings

No significant self-preference bias in RAG evaluations

Factual accuracy influences LLM outputs

Consistent results across datasets and models

Abstract

Recent studies have demonstrated that large language models (LLMs) exhibit significant biases in evaluation tasks, particularly in preferentially rating and favoring self-generated content. However, the extent to which this bias manifests in fact-oriented tasks, especially within retrieval-augmented generation (RAG) frameworks, where keyword extraction and factual accuracy take precedence over stylistic elements, remains unclear. Our study addresses this knowledge gap by simulating two critical phases of the RAG framework. In the first phase, LLMs evaluated human-authored and model-generated passages, emulating the \textit{pointwise reranking phase}. The second phase involves conducting pairwise reading comprehension tests to simulate the \textit{generation phase}. Contrary to previous findings indicating a self-preference in rating tasks, our results reveal no significant self-preference effect in RAG frameworks. Instead, we observe that factual accuracy significantly influences LLMs' output, even in the absence of prior knowledge. These findings are consistent among three common QA datasets (NQ, MARCO, TriviaQA Datasets) and 5 widely adopted language models (GPT-3.5, GPT-4o-mini, Gemini, LLaMA3, and Mistral). Our research contributes to the ongoing discourse on LLM biases and their implications for RAG-based system, offering insights that may inform the development of more robust and unbiased LLM systems.