Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know"

TL;DR

Decomposed prompting helps identify when large language models are uncertain about their knowledge, enabling better error detection without additional training or retrieval, despite not fixing knowledge gaps.

Contribution

The paper shows that disagreement among different prompting regimes effectively signals model uncertainty, leading to a practical abstention method for improving reliability in closed-book QA.

Findings

Disagreement signals correlate with potential errors.

Cross-regime agreement improves error detection performance.

Abstention based on disagreement outperforms standard uncertainty methods.

Abstract

Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations. While decomposed prompting is typically used to improve accuracy, we investigate its impact on reliability. We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks. We find that although accuracy gains from decomposition diminish in frontier models, disagreements between prompting regimes remain highly indicative of potential errors. Because factual knowledge is stable while hallucinations are stochastic, cross-regime agreement provides a precise signal of internal uncertainty. We leverage this signal to implement a training-free abstention policy that requires no retrieval or fine-tuning. Our results show that disagreement-based abstention outperforms standard uncertainty baselines as an error detector, improving both F1 and AUROC across settings. This demonstrates that decomposition-based prompting can serve as a practical diagnostic probe for model reliability in closed-book QA.