Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation

TL;DR

This paper presents a fast, reliable, and well-calibrated uncertainty estimation method for LLM judges using linear probes trained with a Brier score, outperforming existing techniques in calibration and efficiency.

Contribution

Introducing a linear probe approach with Brier score loss for uncertainty estimation that requires no additional training and offers superior calibration and efficiency.

Findings

Probes achieve better calibration than existing methods.

Probes are approximately 10 times more computationally efficient.

Probes generalize well to unseen domains and improve high-confidence accuracy.

Abstract

As LLM-based judges become integral to industry applications, obtaining well-calibrated uncertainty estimates efficiently has become critical for production deployment. However, existing techniques, such as verbalized confidence and multi-generation methods, are often either poorly calibrated or computationally expensive. We introduce linear probes trained with a Brier score-based loss to provide calibrated uncertainty estimates from reasoning judges' hidden states, requiring no additional model training. We evaluate our approach on both objective tasks (reasoning, mathematics, factuality, coding) and subjective human preference judgments. Our results demonstrate that probes achieve superior calibration compared to existing methods with $\approx10$x computational savings, generalize robustly to unseen evaluation domains, and deliver higher accuracy on high-confidence predictions. However, probes produce conservative estimates that underperform on easier datasets but may benefit safety-critical deployments prioritizing low false-positive rates. Overall, our work demonstrates that interpretability-based uncertainty estimation provides a practical and scalable plug-and-play solution for LLM judges in production.