Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge

TL;DR

This paper introduces a statistical framework to detect and measure self-bias in large language models acting as evaluators, ensuring more accurate assessments of model performance.

Contribution

The work presents a novel statistical method to identify and quantify self-bias in LLM-based evaluations, accounting for genuine quality differences and model ability variations.

Findings

Some models, like GPT-4o and Claude 3.5 Sonnet, systematically favor their own outputs.

Models exhibit family-bias, favoring outputs from similar models.

The method reliably isolates self-bias even with varying model abilities.

Abstract

Large language models (LLMs) can serve as judges that offer rapid and reliable assessments of other LLM outputs. However, models may systematically assign overly favorable ratings to their own outputs, a phenomenon known as self-bias, which can distort evaluations of true model performance. Previous studies often conflate genuine differences in model quality with bias or incorrectly assume that evaluations from LLMs and humans follow the same rating distributions. In this work, we present a statistical framework that explicitly formalizes assumptions under which self-bias can be identified and estimated. Our method models the difference in the scoring distribution that LLM-as-a-judge assigns to its own completions compared to other models, while accounting for the underlying quality of the completions provided by an independent, third-party judge (e.g., humans). Our method reliably isolates and quantifies self-bias, even when models vary in ability, ensuring that genuine performance differences are not mistaken for self-bias. We conduct an empirical analysis of self-bias on a large dataset (>5000 prompt-completion pairs) consisting of expert human annotations and judgments from nine different LLM judges. We find that some models, such as GPT-4o and Claude 3.5 Sonnet, systematically assign higher scores to their own outputs. These models also display family-bias; systematically assigning higher ratings to outputs produced by other models of the same family. Our findings highlight potential pitfalls of using LLM judges and offer practical guidance to mitigate biases when interpreting automated evaluations.