PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMs

TL;DR

PAVE introduces a validation and editing layer for retrieval-augmented LLMs, improving answer accuracy by explicitly checking and revising responses based on extracted premises.

Contribution

It presents a novel inference-time validation method that decomposes retrieved context into atomic facts, supports answer revision, and enhances evidence-grounded consistency.

Findings

PAVE outperforms simpler baselines in evidence-grounded QA.

Largest gain of 32.7 accuracy points on a span-grounded benchmark.

Explicit premise extraction improves answer support and reliability.

Abstract

Retrieval-augmented language models can retrieve relevant evidence yet still commit to answers before explicitly checking whether the retrieved context supports the conclusion. We present PAVE (Premise-Grounded Answer Validation and Editing), an inference-time validation layer for evidence-grounded question answering. PAVE decomposes retrieved context into question-conditioned atomic facts, drafts an answer, scores how well that draft is supported by the extracted premises, and revises low-support outputs before finalization. The resulting trace makes answer commitment auditable at the level of explicit premises, support scores, and revision decisions. In controlled ablations with a fixed retriever and backbone, PAVE outperforms simpler post-retrieval baselines in two evidence-grounded QA settings, with the largest gain reaching 32.7 accuracy points on a span-grounded benchmark. We view these findings as proof-of-concept evidence that explicit premise extraction plus support-gated revision can strengthen evidence-grounded consistency in retrieval-augmented LLM systems.