LLMs Should Express Uncertainty Explicitly

TL;DR

This paper explores methods for enabling large language models to explicitly express uncertainty, either through confidence scores or uncertainty markers, to reduce overconfidence and improve answer reliability.

Contribution

It introduces post-training techniques for LLMs to signal uncertainty during or after reasoning, enhancing trustworthiness and enabling better integration with retrieval systems.

Findings

Both methods reduce overconfident errors significantly.

End-of-reasoning confidence sharpens existing model structures.

During-reasoning <uncertain> markers teach high-risk reasoning detection.

Abstract

Large language models (LLMs) often produce confident yet incorrect answers, which can lead to risky failures in real-world applications. We study whether post-training can make a model's self-assessment explicit: when the model is uncertain, can it be trained to signal so within its own response? A central design question is where in the response this signal should be exposed -- during reasoning, while the answer is still being formed, or at the end, once the answer has been produced. We study both. For end-of-reasoning self-assessment, we train the model to verbalize a confidence score for its response, with the aim of high confidence on correct answers and low confidence on incorrect ones. For during-reasoning self-assessment, we train the model to emit the marker <uncertain> whenever its current reasoning state appears unreliable. Across factual reasoning tasks, both forms sharply reduce overconfident errors while improving answer quality, and both can be used as triggers for retrieval augmented generation (RAG) to improve the final response. We further analyze their internal mechanisms: end-of-reasoning verbalized confidence sharpens a confidence-related structure already present in the pretrained model, whereas during-reasoning <uncertain> emission teaches the model to mark high-risk reasoning steps, with parameter changes concentrated in the model's late layers.