Read Your Own Mind: Reasoning Helps Surface Self-Confidence Signals in LLMs

TL;DR

This paper demonstrates that explicit reasoning and exploration of the model's generative space improve the reliability of self-reported confidence signals in large language models during question answering tasks.

Contribution

It shows that forcing a chain-of-thought process enhances confidence calibration, highlighting the importance of exploration for trustworthy uncertainty estimation in LLMs.

Findings

Semantic entropy remains reliable for uncertainty estimation.

Chain-of-thought reasoning improves confidence accuracy.

Self-reported confidence correlates with explored generative space.

Abstract

We study the source of uncertainty in DeepSeek R1-32B by analyzing its self-reported verbal confidence on question answering (QA) tasks. In the default answer-then-confidence setting, the model is regularly over-confident, whereas semantic entropy - obtained by sampling many responses - remains reliable. We hypothesize that this is because of semantic entropy's larger test-time compute, which lets us explore the model's predictive distribution. We show that granting DeepSeek the budget to explore its distribution by forcing a long chain-of-thought before the final answer greatly improves its verbal score effectiveness, even on simple fact-retrieval questions that normally require no reasoning. Furthermore, a separate reader model that sees only the chain can reconstruct very similar confidences, indicating the verbal score might be merely a statistic of the alternatives surfaced during reasoning. Our analysis concludes that reliable uncertainty estimation requires explicit exploration of the generative space, and self-reported confidence is trustworthy only after such exploration.