MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation

TL;DR

MedRAGChecker is a framework that verifies the factual accuracy of biomedical claims in RAG outputs by decomposing answers into claims and assessing support using natural language inference and knowledge graphs, improving safety and reliability.

Contribution

It introduces a claim-level verification framework combining NLI and biomedical knowledge graphs for biomedical RAG, enabling scalable, reliable detection of unsupported or contradictory claims.

Findings

Effectively flags unsupported claims in biomedical RAG outputs.

Reveals safety-critical claim errors across different generators.

Provides answer-level diagnostics for retrieval and generation failures.

Abstract

Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce MedRAGChecker, a claim-level verification and diagnostic framework for biomedical RAG. Given a question, retrieved evidence, and a generated answer, MedRAGChecker decomposes the answer into atomic claims and estimates claim support by combining evidence-grounded natural language inference (NLI) with biomedical knowledge-graph (KG) consistency signals. Aggregating claim decisions yields answer-level diagnostics that help disentangle retrieval and generation failures, including faithfulness, under-evidence, contradiction, and safety-critical error rates. To enable scalable evaluation, we distill the pipeline into compact biomedical models and use an ensemble verifier with class-specific reliability weighting. Experiments on four biomedical QA benchmarks show that MedRAGChecker reliably flags unsupported and contradicted claims and reveals distinct risk profiles across generators, particularly on safety-critical biomedical relations.