TL;DR
VerifAI is an open-source biomedical question answering system that combines retrieval, generation, and claim verification to ensure factual accuracy and transparency.
Contribution
It introduces a modular system integrating retrieval, generative, and verification components with a novel claim validation mechanism, outperforming existing methods.
Findings
VerifAI achieves a MAP@10 of 42.7% on biomedical IR tasks.
It significantly reduces hallucinated citations compared to baselines.
VerifAI outperforms GPT-4 in claim verification accuracy on HealthVer benchmark.
Abstract
We introduce VerifAI, an open-source expert system for biomedical question answering that integrates retrieval-augmented generation (RAG) with a novel post-hoc claim verification mechanism. Unlike standard RAG systems, VerifAI ensures factual consistency by decomposing generated answers into atomic claims and validating them against retrieved evidence using a fine-tuned natural language inference (NLI) engine. The system comprises three modular components: (1) a hybrid Information Retrieval (IR) module optimized for biomedical queries (MAP@10 of 42.7%), (2) a citation-aware Generative Component fine-tuned on a custom dataset to produce referenced answers, and (3) a Verification Component that detects hallucinations with state-of-the-art accuracy, outperforming GPT-4 on the HealthVer benchmark. Evaluations demonstrate that VerifAI significantly reduces hallucinated citations compared to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
