META-RAG: Meta-Analysis-Inspired Evidence-Re-Ranking Method for Retrieval-Augmented Generation in Evidence-Based Medicine

TL;DR

This paper introduces META-RAG, a novel evidence re-ranking method inspired by meta-analysis to improve the quality of medical evidence retrieved for LLMs in evidence-based medicine, significantly enhancing diagnostic accuracy.

Contribution

The paper proposes a meta-analysis-inspired re-ranking approach for evidence retrieval in RAG, improving evidence quality and diagnostic accuracy in EBM tasks.

Findings

Achieved up to 11.4% accuracy improvement in experiments.

Successfully filtered higher-quality evidence from PubMed.

Enhanced reliability of evidence used by LLMs in medical diagnosis.

Abstract

Evidence-based medicine (EBM) holds a crucial role in clinical application. Given suitable medical articles, doctors effectively reduce the incidence of misdiagnoses. Researchers find it efficient to use large language models (LLMs) techniques like RAG for EBM tasks. However, the EBM maintains stringent requirements for evidence, and RAG applications in EBM struggle to efficiently distinguish high-quality evidence. Therefore, inspired by the meta-analysis used in EBM, we provide a new method to re-rank and filter the medical evidence. This method presents multiple principles to filter the best evidence for LLMs to diagnose. We employ a combination of several EBM methods to emulate the meta-analysis, which includes reliability analysis, heterogeneity analysis, and extrapolation analysis. These processes allow the users to retrieve the best medical evidence for the LLMs. Ultimately, we evaluate these high-quality articles and show an accuracy improvement of up to 11.4% in our experiments and results. Our method successfully enables RAG to extract higher-quality and more reliable evidence from the PubMed dataset. This work can reduce the infusion of incorrect knowledge into responses and help users receive more effective replies.