Are LLM Decisions Faithful to Verbal Confidence?
Jiawei Wang, Yanfei Zhou, Siddartha Devic, Deqing Fu

TL;DR
This paper introduces RiskEval, a framework to assess if large language models' verbal confidence aligns with their decision-making, revealing a disconnect between expressed confidence and strategic abstention under varying error penalties.
Contribution
The paper presents RiskEval, a novel evaluation framework that tests whether LLMs' confidence scores influence their abstention behavior in risk-sensitive scenarios.
Findings
Models are not cost-aware in verbal confidence expression.
Models do not strategically abstain under high penalties.
Verbal confidence scores are insufficient for trustworthy decision-making.
Abstract
Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decision making of the model. To test this, we introduce : a framework designed to evaluate whether models adjust their abstention policies in response to varying error penalties. Our evaluation of several frontier models reveals a critical dissociation: models are neither cost-aware when articulating their verbal confidence, nor strategically responsive when deciding whether to engage or abstain under high-penalty conditions. Even when extreme penalties render frequent abstention the mathematically optimal strategy, models almost never abstain, resulting in utility collapse. This indicates that calibrated verbal confidence scores may not be sufficient to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEthics and Social Impacts of AI · Explainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education
