MedBioRAG: Semantic Search and Retrieval-Augmented Generation with Large Language Models for Medical and Biological QA

TL;DR

MedBioRAG is a novel retrieval-augmented large language model that significantly improves biomedical question-answering by combining semantic search, document retrieval, and fine-tuning, outperforming existing models on multiple benchmark datasets.

Contribution

We introduce MedBioRAG, a new biomedical QA model that integrates semantic search, document retrieval, and supervised fine-tuning, achieving state-of-the-art results.

Findings

Outperforms previous state-of-the-art models on benchmark datasets.

Improves NDCG and MRR scores for document retrieval.

Achieves higher accuracy and ROUGE scores in biomedical QA tasks.

Abstract

Recent advancements in retrieval-augmented generation (RAG) have significantly enhanced the ability of large language models (LLMs) to perform complex question-answering (QA) tasks. In this paper, we introduce MedBioRAG, a retrieval-augmented model designed to improve biomedical QA performance through a combination of semantic and lexical search, document retrieval, and supervised fine-tuning. MedBioRAG efficiently retrieves and ranks relevant biomedical documents, enabling precise and context-aware response generation. We evaluate MedBioRAG across text retrieval, close-ended QA, and long-form QA tasks using benchmark datasets such as NFCorpus, TREC-COVID, MedQA, PubMedQA, and BioASQ. Experimental results demonstrate that MedBioRAG outperforms previous state-of-the-art (SoTA) models and the GPT-4o base model in all evaluated tasks. Notably, our approach improves NDCG and MRR scores for document retrieval, while achieving higher accuracy in close-ended QA and ROUGE scores in long-form QA. Our findings highlight the effectiveness of semantic search-based retrieval and LLM fine-tuning in biomedical applications.