Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution

TL;DR

This paper identifies overconfidence in large language models used as judges, introduces a new metric to measure confidence calibration, and proposes an ensemble framework to improve reliability and risk-awareness in evaluations.

Contribution

It systematically diagnoses overconfidence in LLMs, introduces TH-Score for calibration assessment, and presents LLM-as-a-Fuser, a novel ensemble method for trustworthy, confidence-driven evaluation.

Findings

Overconfidence significantly reduces LLM evaluation reliability.

TH-Score effectively measures confidence-accuracy alignment.

LFM-as-a-Fuser improves calibration and adaptive evaluation performance.

Abstract

Large Language Models (LLMs) are widely used as automated judges, where practical value depends on both accuracy and trustworthy, risk-aware judgments. Existing approaches predominantly focus on accuracy, overlooking the necessity of well-calibrated confidence, which is vital for adaptive and reliable evaluation pipelines. In this work, we advocate a shift from accuracy-centric evaluation to confidence-driven, risk-aware LLM-as-a-Judge systems, emphasizing the necessity of well-calibrated confidence for trustworthy and adaptive evaluation. We systematically identify the Overconfidence Phenomenon in current LLM-as-a-Judges, where predicted confidence significantly overstates actual correctness, undermining reliability in practical deployment. To quantify this phenomenon, we introduce TH-Score, a novel metric measuring confidence-accuracy alignment. Furthermore, we propose LLM-as-a-Fuser, an ensemble framework that transforms LLMs into reliable, risk-aware evaluators. Extensive experiments demonstrate that our approach substantially improves calibration and enables adaptive, confidence-driven evaluation pipelines, achieving superior reliability and accuracy compared to existing baselines.