Beyond the Surface: Measuring Self-Preference in LLM Judgments

TL;DR

This paper introduces the DBG score, a novel method for accurately measuring self-preference bias in large language models by using gold judgments to separate bias from response quality, enabling more reliable assessments.

Contribution

The paper proposes the DBG score, which isolates self-preference bias from response quality using gold judgments, and applies it to analyze factors affecting bias in LLMs.

Findings

DBG score effectively separates bias from response quality.

Self-preference bias varies with model size and reasoning ability.

Response style and post-training data influence bias levels.

Abstract

Recent studies show that large language models (LLMs) exhibit self-preference bias when serving as judges, meaning they tend to favor their own responses over those generated by other models. Existing methods typically measure this bias by calculating the difference between the scores a judge model assigns to its own responses and those it assigns to responses from other models. However, this approach conflates self-preference bias with response quality, as higher-quality responses from the judge model may also lead to positive score differences, even in the absence of bias. To address this issue, we introduce gold judgments as proxies for the actual quality of responses and propose the DBG score, which measures self-preference bias as the difference between the scores assigned by the judge model to its own responses and the corresponding gold judgments. Since gold judgments reflect true response quality, the DBG score mitigates the confounding effect of response quality on bias measurement. Using the DBG score, we conduct comprehensive experiments to assess self-preference bias across LLMs of varying versions, sizes, and reasoning abilities. Additionally, we investigate two factors that influence and help alleviate self-preference bias: response text style and the post-training data of judge models. Finally, we explore potential underlying mechanisms of self-preference bias from an attention-based perspective. Our code and data are available at https://github.com/zhiyuanc2001/self-preference.