M-Eval: A Heterogeneity-Based Framework for Multi-evidence Validation in Medical RAG Systems

TL;DR

M-Eval introduces a heterogeneity-based framework that improves validation of medical RAG responses by detecting factual errors and assessing evidence reliability, enhancing system accuracy and trustworthiness.

Contribution

This paper presents M-Eval, a novel heterogeneity analysis approach for verifying factual correctness and evidence reliability in medical RAG systems, addressing hallucinations and misinformation.

Findings

Up to 23.31% accuracy improvement across LLMs.

Effective detection of factual errors and evidence inconsistencies.

Enhanced reliability of medical RAG responses.

Abstract

Retrieval-augmented Generation (RAG) has demonstrated potential in enhancing medical question-answering systems through the integration of large language models (LLMs) with external medical literature. LLMs can retrieve relevant medical articles to generate more professional responses efficiently. However, current RAG applications still face problems. They generate incorrect information, such as hallucinations, and they fail to use external knowledge correctly. To solve these issues, we propose a new method named M-Eval. This method is inspired by the heterogeneity analysis approach used in Evidence-Based Medicine (EBM). Our approach can check for factual errors in RAG responses using evidence from multiple sources. First, we extract additional medical literature from external knowledge bases. Then, we retrieve the evidence documents generated by the RAG system. We use heterogeneity analysis to check whether the evidence supports different viewpoints in the response. In addition to verifying the accuracy of the response, we also assess the reliability of the evidence provided by the RAG system. Our method shows an improvement of up to 23.31% accuracy across various LLMs. This work can help detect errors in current RAG-based medical systems. It also makes the applications of LLMs more reliable and reduces diagnostic errors.