Certainty robustness: Evaluating LLM stability under self-challenging prompts

TL;DR

This paper introduces a new benchmark to evaluate how large language models respond to self-challenging prompts, revealing differences in their stability and trustworthiness during interactive questioning.

Contribution

The paper proposes the Certainty Robustness Benchmark, a two-turn evaluation framework for assessing LLM stability and adaptability under challenging prompts, highlighting a new dimension of model evaluation.

Findings

Some models abandon correct answers under challenge.

Other models resist challenge and align confidence with correctness.

Substantial differences in interactive reliability among models.

Abstract

Large language models (LLMs) often present answers with high apparent confidence despite lacking an explicit mechanism for reasoning about certainty or truth. While existing benchmarks primarily evaluate single-turn accuracy, truthfulness or confidence calibration, they do not capture how models behave when their responses are challenged in interactive settings. We introduce the Certainty Robustness Benchmark, a two-turn evaluation framework that measures how LLMs balance stability and adaptability under self-challenging prompts such as uncertainty ("Are you sure?") and explicit contradiction ("You are wrong!"), alongside numeric confidence elicitation. Using 200 reasoning and mathematics questions from LiveBench, we evaluate four state-of-the-art LLMs and distinguish between justified self-corrections and unjustified answer changes. Our results reveal substantial differences in interactive reliability that are not explained by baseline accuracy alone: some models abandon correct answers under conversational pressure, while others demonstrate strong resistance to challenge and better alignment between confidence and correctness. These findings identify certainty robustness as a distinct and critical dimension of LLM evaluation, with important implications for alignment, trustworthiness and real-world deployment.