Self-Preference Bias in LLM-as-a-Judge

TL;DR

This paper introduces a new metric to quantify self-preference bias in LLM evaluators, revealing that models like GPT-4 favor familiar, low-perplexity outputs, which can skew automated dialogue system assessments.

Contribution

The paper presents a novel quantitative measure for self-preference bias and analyzes its root cause related to output perplexity in LLM evaluations.

Findings

GPT-4 exhibits significant self-preference bias.

LLMs prefer outputs with lower perplexity, regardless of origin.

Bias can skew automated evaluation results.

Abstract

Automated evaluation leveraging large language models (LLMs), commonly referred to as LLM evaluators or LLM-as-a-judge, has been widely used in measuring the performance of dialogue systems. However, the self-preference bias in LLMs has posed significant risks, including promoting specific styles or policies intrinsic to the LLMs. Despite the importance of this issue, there is a lack of established methods to measure the self-preference bias quantitatively, and its underlying causes are poorly understood. In this paper, we introduce a novel quantitative metric to measure the self-preference bias. Our experimental results demonstrate that GPT-4 exhibits a significant degree of self-preference bias. To explore the causes, we hypothesize that LLMs may favor outputs that are more familiar to them, as indicated by lower perplexity. We analyze the relationship between LLM evaluations and the perplexities of outputs. Our findings reveal that LLMs assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated. This suggests that the essence of the bias lies in perplexity and that the self-preference bias exists because LLMs prefer texts more familiar to them.