VeriTrail: Closed-Domain Hallucination Detection with Traceability
Dasha Metropolitansky, Jonathan Larson
TL;DR
VeriTrail is a novel method for detecting hallucinations in language models that provides traceability of where unsubstantiated content originates, especially in complex multi-step generation processes.
Contribution
This paper introduces VeriTrail, the first closed-domain hallucination detection approach that offers traceability and the first datasets with intermediate outputs and human annotations.
Findings
VeriTrail outperforms baseline methods in hallucination detection.
The datasets include intermediate outputs and human annotations.
Traceability improves understanding of hallucination sources.
Abstract
Even when instructed to adhere to source material, language models often generate unsubstantiated content - a phenomenon known as "closed-domain hallucination." This risk is amplified in processes with multiple generative steps (MGS), compared to processes with a single generative step (SGS). However, due to the greater complexity of MGS processes, we argue that detecting hallucinations in their final outputs is necessary but not sufficient: it is equally important to trace where hallucinated content was likely introduced and how faithful content may have been derived from the source material through intermediate outputs. To address this need, we present VeriTrail, the first closed-domain hallucination detection method designed to provide traceability for both MGS and SGS processes. We also introduce the first datasets to include all intermediate outputs as well as human annotations of…
Peer Reviews
Decision·ICLR 2026 Poster
+ The paper proposes a new framework for hallucination detection in multi-generation process. + It curates two datasets for faithfulness evaluation in MGS based on previous benchmarks. + The proposed VeriTrail framework shows a substaintial improvement over previous baseline.
+ Missing intermediate results. The ablation analysis In Appendix E shows that evidence selection using language model plays an important role in final performance so it would be necessary to include the accuracy of the evidence selection in evaluation. + Since VeriTrail uses stage-by-stage verfiication and does not have to always checks the source materials if the verfiication process exits early, I would suggest analyzing the API token cost of VeriTrail and comparing with the baseline method.
The paper tackles the critical problem of error propagation in MGS processes. The focus on "traceability" beyond simple detection is a key conceptual contribution. The method is tested against strong, comprehensive baselines.
A primary weakness of this method is its recursive reliance on Large Language Models (LLMs). Core steps of VeriTrail, including sub-claim decomposition, evidence selection, and final verdict generation, are all dependent on LLMs. This creates a fundamental "verifier's dilemma": the LLM used for verification may itself hallucinate or make faulty inferences. The paper's own error analysis in Appendix G concedes that the verification model itself can make "invalid inferences" or improperly use "pa
1) The paper studies a pertinent problem of detecting hallucinations in the generations of autoregressive LLMs, while also focusing on provenance tracing and error localization with multiple-generation steps, which is highly relevant given complex, multi-agent systems that have gained prominence in recent times. By identifying the intermediate stage at which hallucinations are introduced in a closed-domain setting, specific and actionable modes of improvement can be identified and undertaken in
1) The proposed method VeriTrail is not simple, and requires a fairly complex setting up of the generative process as a DAG, and ensure that each intermediate output is carefully produced and analyzed in a multi-step manner with verification by an LLM. This also places a constraint on its scalability in practice for real-time applications, given the need for multiple LLM calls for each claim/sub-claim in the final output. Could the authors also kindly provide some metrics with respect to the act
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Adversarial Robustness in Machine Learning · Mental Health via Writing