Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applications

TL;DR

This paper introduces MedOmniKB and a source planning optimization method to improve multi-source knowledge retrieval for large language models in medical applications, reducing hallucinations and enhancing accuracy.

Contribution

It presents a comprehensive medical knowledge repository and a novel source planning approach that aligns source attributes with model expectations, improving multi-source utilization.

Findings

Significant improvement in multi-source planning performance

Achieved state-of-the-art results in medical knowledge retrieval

Enhanced model ability to leverage diverse sources effectively

Abstract

Large language models hold promise for addressing medical challenges, such as medical diagnosis reasoning, research knowledge acquisition, clinical decision-making, and consumer health inquiry support. However, they often generate hallucinations due to limited medical knowledge. Incorporating external knowledge is therefore critical, which necessitates multi-source knowledge acquisition. We address this challenge by framing it as a source planning problem, which is to formulate context-appropriate queries tailored to the attributes of diverse sources. Existing approaches either overlook source planning or fail to achieve it effectively due to misalignment between the model's expectation of the sources and their actual content. To bridge this gap, we present MedOmniKB, a repository comprising multigenre and multi-structured medical knowledge sources. Leveraging these sources, we propose the Source Planning Optimisation method, which enhances multi-source utilisation. Our approach involves enabling an expert model to explore and evaluate potential plans while training a smaller model to learn source alignment. Experimental results demonstrate that our method substantially improves multi-source planning performance, enabling the optimised small model to achieve state-of-the-art results in leveraging diverse medical knowledge sources.