Black-box Uncertainty Quantification Method for LLM-as-a-Judge

TL;DR

This paper introduces a novel black-box uncertainty quantification method for LLMs acting as evaluators, improving the trustworthiness and consistency of their assessments across various benchmarks.

Contribution

The paper presents a new black-box approach to quantify uncertainty in LLM evaluations, addressing a gap in trustworthiness and reliability in LLM-as-a-Judge systems.

Findings

Strong correlation between evaluation accuracy and uncertainty scores

Method enhances reliability of LLM-based assessments

Effective across multiple benchmark datasets

Abstract

LLM-as-a-Judge is a widely used method for evaluating the performance of Large Language Models (LLMs) across various tasks. We address the challenge of quantifying the uncertainty of LLM-as-a-Judge evaluations. While uncertainty quantification has been well-studied in other domains, applying it effectively to LLMs poses unique challenges due to their complex decision-making capabilities and computational demands. In this paper, we introduce a novel method for quantifying uncertainty designed to enhance the trustworthiness of LLM-as-a-Judge evaluations. The method quantifies uncertainty by analyzing the relationships between generated assessments and possible ratings. By cross-evaluating these relationships and constructing a confusion matrix based on token probabilities, the method derives labels of high or low uncertainty. We evaluate our method across multiple benchmarks, demonstrating a strong correlation between the accuracy of LLM evaluations and the derived uncertainty scores. Our findings suggest that this method can significantly improve the reliability and consistency of LLM-as-a-Judge evaluations.