MedKP: Medical Dialogue with Knowledge Enhancement and Clinical Pathway Encoding

TL;DR

MedKP enhances medical dialogue generation by integrating external medical knowledge and clinical pathways into LLMs, significantly reducing hallucinations and improving response accuracy in real-world datasets.

Contribution

This work introduces MedKP, a novel framework combining knowledge graphs and clinical pathway encoding to improve medical dialogue generation with LLMs.

Findings

Outperforms multiple baselines on MedDG and KaMed datasets.

Reduces hallucination rates in medical responses.

Achieves state-of-the-art performance in medical dialogue tasks.

Abstract

With appropriate data selection and training techniques, Large Language Models (LLMs) have demonstrated exceptional success in various medical examinations and multiple-choice questions. However, the application of LLMs in medical dialogue generation-a task more closely aligned with actual medical practice-has been less explored. This gap is attributed to the insufficient medical knowledge of LLMs, which leads to inaccuracies and hallucinated information in the generated medical responses. In this work, we introduce the Medical dialogue with Knowledge enhancement and clinical Pathway encoding (MedKP) framework, which integrates an external knowledge enhancement module through a medical knowledge graph and an internal clinical pathway encoding via medical entities and physician actions. Evaluated with comprehensive metrics, our experiments on two large-scale, real-world online medical consultation datasets (MedDG and KaMed) demonstrate that MedKP surpasses multiple baselines and mitigates the incidence of hallucinations, achieving a new state-of-the-art. Extensive ablation studies further reveal the effectiveness of each component of MedKP. This enhancement advances the development of reliable, automated medical consultation responses using LLMs, thereby broadening the potential accessibility of precise and real-time medical assistance.