MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph

TL;DR

MedKGent introduces a novel framework utilizing large language models to construct a temporally evolving medical knowledge graph from PubMed abstracts, improving knowledge integration and supporting advanced biomedical applications.

Contribution

This work presents MedKGent, the first LLM-based framework for incremental, temporally-aware construction of large-scale medical knowledge graphs from dynamic literature data.

Findings

Constructed a KG with over 156k entities and 3 million triples.

Achieved nearly 90% accuracy in knowledge extraction validated by experts.

Enhanced medical question answering performance using the KG in retrieval-augmented generation.

Abstract

The rapid expansion of medical literature presents growing challenges for structuring and integrating domain knowledge at scale. Knowledge Graphs (KGs) offer a promising solution by enabling efficient retrieval, automated reasoning, and knowledge discovery. However, current KG construction methods often rely on supervised pipelines with limited generalizability or naively aggregate outputs from Large Language Models (LLMs), treating biomedical corpora as static and ignoring the temporal dynamics and contextual uncertainty of evolving knowledge. To address these limitations, we introduce MedKGent, a LLM agent framework for constructing temporally evolving medical KGs. Leveraging over 10 million PubMed abstracts published between 1975 and 2023, we simulate the emergence of biomedical knowledge via a fine-grained daily time series. MedKGent incrementally builds the KG in a day-by-day manner using two specialized agents powered by the Qwen2.5-32B-Instruct model. The Extractor Agent identifies knowledge triples and assigns confidence scores via sampling-based estimation, which are used to filter low-confidence extractions and inform downstream processing. The Constructor Agent incrementally integrates the retained triples into a temporally evolving graph, guided by confidence scores and timestamps to reinforce recurring knowledge and resolve conflicts. The resulting KG contains 156,275 entities and 2,971,384 relational triples. Quality assessments by two SOTA LLMs and three domain experts demonstrate an accuracy approaching 90%, with strong inter-rater agreement. To evaluate downstream utility, we conduct RAG across seven medical question answering benchmarks using five leading LLMs, consistently observing significant improvements over non-augmented baselines. Case studies further demonstrate the KG's value in literature-based drug repurposing via confidence-aware causal inference.