Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution

TL;DR

This paper introduces a method for LLMs to verbalize probability distributions, improving confidence estimation and reasoning, with significant efficiency gains and broad applicability across tasks.

Contribution

It demonstrates that predicting verbalized probability distributions enhances reasoning and confidence estimation, outperforming previous methods and reducing computational costs.

Findings

Achieves higher reasoning efficacy during inference-time scaling.

Saves nearly 6× the computation to reach optimal confidence scores.

Shows advantages across multiple LLMs and tasks.

Abstract

Knowing the reliability of a model's response is essential in practical applications. Given the strong generation capabilities of large language models (LLMs), research has focused on generating verbalized confidence. This approach is further enhanced by integrating chain-of-thought reasoning, which provides logical and transparent estimates. However, how reasoning strategies affect the estimated confidence remains under-explored. In this work, we demonstrate that predicting a verbalized probability distribution effectively promotes reasoning for confidence estimation. It requires an LLM to consider all possible answers rather than relying on a single guess, and the requirement of producing a distribution elicits more careful confidence assignment. We conduct systematic experiments comparing different verbalization-based methods across multiple LLMs and tasks. Our method consistently shows advantages, whether in the simple prompting setup or after optimization via reinforcement learning (RL). Notably, it achieves higher reasoning efficacy during inference-time scaling, saving nearly 6$\times$ the computation to reach the best Brier score of the strongest baseline on MMLU-Pro. Additionally, we reveal its limitations on specific tasks and discuss possible solutions for broader applicability.