How to Correctly Report LLM-as-a-Judge Evaluations

TL;DR

This paper introduces a bias-correcting framework for using large language models as evaluators, providing statistically sound confidence intervals and adaptive calibration strategies, outperforming human evaluation under certain conditions.

Contribution

It presents a novel plug-in method that corrects LLM evaluation bias, quantifies uncertainty, and remains robust under distribution shifts, advancing automated model assessment.

Findings

Framework constructs confidence intervals accounting for dataset uncertainties

Adaptive calibration improves interval tightness and evaluation reliability

Method outperforms human evaluation under specific parameter regimes

Abstract

Large language models (LLMs) are widely used as scalable evaluators of model responses in lieu of human annotators. However, imperfect sensitivity and specificity of the LLM judges induce bias in naive evaluation scores. We propose a simple plug-in framework that corrects this bias and enables statistically principled uncertainty quantification. Our framework constructs confidence intervals that account for uncertainty from both the test dataset and a human-labeled calibration dataset. Additionally, it uses an adaptive strategy to allocate calibration samples for tighter intervals. Importantly, we characterize parameter regimes defined by the true evaluation score and the LLM judge's sensitivity and specificity in which our LLM-based evaluation yields more reliable estimates than human-only evaluation. Moreover, we show that our framework remains unbiased under distribution shift between the test and calibration datasets, in contrast to existing approaches.