Scientific QA System with Verifiable Answers
Adela Ljaji\'c, Milo\v{s} Ko\v{s}prdi\'c, Bojana Ba\v{s}aragin, Darija, Medvecki, Lorenzo Cassano, Nikola Milo\v{s}evi\'c

TL;DR
This paper presents VerifAI, an open-source scientific QA system that combines retrieval, generation, and verification modules to produce and validate factual scientific answers, reducing hallucinations and increasing trust.
Contribution
The paper introduces VerifAI, a novel system integrating retrieval, generative, and verification components specifically designed for scientific question answering.
Findings
Effective retrieval of scientific papers from PubMed.
Generation of claims with references using fine-tuned models.
Verification engine reduces hallucinations in generated answers.
Abstract
In this paper, we introduce the VerifAI project, a pioneering open-source scientific question-answering system, designed to provide answers that are not only referenced but also automatically vetted and verifiable. The components of the system are (1) an Information Retrieval system combining semantic and lexical search techniques over scientific papers (PubMed), (2) a Retrieval-Augmented Generation (RAG) module using fine-tuned generative model (Mistral 7B) and retrieved articles to generate claims with references to the articles from which it was derived, and (3) a Verification engine, based on a fine-tuned DeBERTa and XLM-RoBERTa models on Natural Language Inference task using SciFACT dataset. The verification engine cross-checks the generated claim and the article from which the claim was derived, verifying whether there may have been any hallucinations in generating the claim. By…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScientific Computing and Data Management · Semantic Web and Ontologies
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Attention Dropout · Linear Warmup With Linear Decay · Residual Connection · Adam · Dropout · Byte Pair Encoding · Layer Normalization · Linear Layer
