Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation

TL;DR

This paper introduces a domain-specific retrieval-augmented generation framework that enhances factual accuracy and verification in biomedical question answering by integrating explicit reasoning, rationale grounding, and a detailed verification taxonomy.

Contribution

It proposes a novel RAG architecture with explicit rationale generation and a verification taxonomy, improving factuality and interpretability in biomedical QA tasks.

Findings

Explicit rationale generation improves accuracy over baseline RAG.

Dynamic demonstration selection enhances few-shot performance.

The approach achieves high accuracy with a smaller model size.

Abstract

Retrieval-Augmented Generation (RAG) significantly improves the factuality of Large Language Models (LLMs), yet standard pipelines often lack mechanisms to verify inter- mediate reasoning, leaving them vulnerable to hallucinations in high-stakes domains. To address this, we propose a domain-specific RAG framework that integrates explicit rea- soning and faithfulness verification. Our architecture augments standard retrieval with neural query rewriting, BGE-based cross-encoder reranking, and a rationale generation module that grounds sub-claims in specific evidence spans. We further introduce an eight-category verification taxonomy that enables fine-grained assessment of rationale faithfulness, distinguishing between explicit and implicit support patterns to facilitate structured error diagnosis. We evaluate this framework on the BioASQ and PubMedQA benchmarks, specifically analyzing the impact of dynamic in-context learning and rerank- ing under constrained token budgets. Experiments demonstrate that explicit rationale generation improves accuracy over vanilla RAG baselines, while dynamic demonstration selection combined with robust reranking yields further gains in few-shot settings. Using Llama-3-8B-Instruct, our approach achieves 89.1% on BioASQ-Y/N and 73.0% on Pub- MedQA, competitive with systems using significantly larger models. Additionally, we perform a pilot study combining human expert assessment with LLM-based verification to explore how explicit rationale generation improves system transparency and enables more detailed diagnosis of retrieval failures in biomedical question answering.