Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge

TL;DR

This paper introduces AMG-RAG, a framework that automates medical knowledge graph construction and updating, integrating reasoning and external evidence retrieval to improve medical question-answering accuracy and interpretability.

Contribution

The paper presents AMG-RAG, a novel automated system for building and updating medical knowledge graphs that enhances LLM-based medical question-answering with real-time evidence integration.

Findings

Achieves 74.1% F1 on MEDQA benchmark.

Attains 66.34% accuracy on MEDMCQA.

Outperforms larger models without extra computational cost.

Abstract

Large Language Models (LLMs) have significantly advanced medical question-answering by leveraging extensive clinical data and medical literature. However, the rapid evolution of medical knowledge and the labor-intensive process of manually updating domain-specific resources pose challenges to the reliability of these systems. To address this, we introduce Agentic Medical Graph-RAG (AMG-RAG), a comprehensive framework that automates the construction and continuous updating of medical knowledge graphs, integrates reasoning, and retrieves current external evidence, such as PubMed and WikiSearch. By dynamically linking new findings and complex medical concepts, AMG-RAG not only improves accuracy but also enhances interpretability in medical queries. Evaluations on the MEDQA and MEDMCQA benchmarks demonstrate the effectiveness of AMG-RAG, achieving an F1 score of 74.1 percent on MEDQA and an accuracy of 66.34 percent on MEDMCQA, outperforming both comparable models and those 10 to 100 times larger. Notably, these improvements are achieved without increasing computational overhead, highlighting the critical role of automated knowledge graph generation and external evidence retrieval in delivering up-to-date, trustworthy medical insights.