Agentic Uncertainty Reveals Agentic Overconfidence

TL;DR

This paper investigates agentic uncertainty in AI agents, revealing widespread overconfidence and showing that pre-execution assessments with less information can sometimes outperform post-execution reviews in predicting success.

Contribution

It introduces the concept of agentic uncertainty, demonstrates pervasive overconfidence in AI agents, and finds that certain assessment methods, like adversarial prompting, improve calibration.

Findings

Agents exhibit overconfidence, predicting success rates much higher than actual.

Pre-execution assessments with less information can outperform post-execution reviews.

Adversarial prompting as bug-finding improves calibration.

Abstract

Can AI agents predict whether they will succeed at a task? We study agentic uncertainty by eliciting success probability estimates before, during, and after task execution. All results exhibit agentic overconfidence: some agents that succeed only 22% of the time predict 77% success. Counterintuitively, pre-execution assessment with strictly less information tends to yield better discrimination than standard post-execution review, though differences are not always significant. Adversarial prompting reframing assessment as bug-finding achieves the best calibration.