Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge

TL;DR

This paper investigates language bias in Large Language Model-based judges, revealing significant disparities across languages and showing that bias is not solely due to perplexity, highlighting challenges in fair AI evaluation.

Contribution

The study systematically analyzes language bias in LLM-as-a-judge, identifying performance disparities and the influence of answer language, and explores the relationship with perplexity bias.

Findings

European languages outperform African languages in same-language judging

Models favor English answers in inter-language comparisons

Language bias is only partially explained by perplexity

Abstract

Recent advances in Large Language Models (LLMs) have incentivized the development of LLM-as-a-judge, an application of LLMs where they are used as judges to decide the quality of a certain piece of text given a certain context. However, previous studies have demonstrated that LLM-as-a-judge can be biased towards different aspects of the judged texts, which often do not align with human preference. One of the identified biases is language bias, which indicates that the decision of LLM-as-a-judge can differ based on the language of the judged texts. In this paper, we study two types of language bias in pairwise LLM-as-a-judge: (1) performance disparity between languages when the judge is prompted to compare options from the same language, and (2) bias towards options written in major languages when the judge is prompted to compare options of two different languages. We find that for same-language judging, there exist significant performance disparities across language families, with European languages consistently outperforming African languages, and this bias is more pronounced in culturally-related subjects. For inter-language judging, we observe that most models favor English answers, and that this preference is influenced more by answer language than question language. Finally, we investigate whether language bias is in fact caused by low-perplexity bias, a previously identified bias of LLM-as-a-judge, and we find that while perplexity is slightly correlated with language bias, language bias cannot be fully explained by perplexity only.