When Is Enough Not Enough? Illusory Completion in Search Agents

TL;DR

This paper investigates the reasoning reliability of search agents on multi-constraint problems, revealing frequent illusory completion and proposing the Epistemic Ledger framework to diagnose and improve their constraint verification capabilities.

Contribution

It introduces the Epistemic Ledger framework for tracking evidence and beliefs during reasoning, and demonstrates that explicit constraint-state tracking improves agent accuracy and reduces errors.

Findings

Illusory completion occurs frequently in multi-constraint reasoning.

The Epistemic Ledger effectively diagnoses reasoning failures.

Explicit constraint tracking improves accuracy and reduces underverification.

Abstract

Recent search agents leverage multi-turn reasoning and search tools to achieve strong performance on multi-hop and long-horizon benchmarks. Yet it remains unclear whether they reliably reason across all requirements by tracking, verifying, and maintaining multiple conditions in these questions. We study this capability under multi-constraint problems, where valid answers must satisfy several constraints simultaneously. We find that illusory completion frequently occurs, wherein agents believe tasks are complete despite unresolved or violated constraints, leading to underverified answers. To diagnose this behavior, we introduce the Epistemic Ledger, an evaluation framework that tracks evidential support and agents' beliefs for each constraint throughout multi-turn reasoning. Our analysis reveals four recurring failure patterns: bare assertions, overlooked refutations, stagnation, and premature exit. Motivated by these findings, we examine whether explicit constraint-state tracking during execution mitigates these failures via LiveLedger, an inference-time tracker. This simple intervention consistently improves performance, substantially reducing underverified answers (by up to 26.5%) and improving overall accuracy (by up to 11.6%) on multi-constraint problems.