Conversational Disease Diagnosis via External Planner-Controlled Large Language Models

TL;DR

This paper introduces a novel LLM-based medical diagnostic system that uses external planners to improve proactive patient data collection and diagnosis accuracy, demonstrating promising results with real medical data.

Contribution

The study presents a dual-planner system that enhances LLM diagnostic capabilities by integrating reinforcement learning and medical guideline parsing, advancing AI's role in clinical diagnosis.

Findings

High accuracy in disease screening tasks

Effective differential diagnosis performance

Successful use of real patient data for evaluation

Abstract

The development of large language models (LLMs) has brought unprecedented possibilities for artificial intelligence (AI) based medical diagnosis. However, the application perspective of LLMs in real diagnostic scenarios is still unclear because they are not adept at collecting patient data proactively. This study presents a LLM-based diagnostic system that enhances planning capabilities by emulating doctors. Our system involves two external planners to handle planning tasks. The first planner employs a reinforcement learning approach to formulate disease screening questions and conduct initial diagnoses. The second planner uses LLMs to parse medical guidelines and conduct differential diagnoses. By utilizing real patient electronic medical record data, we constructed simulated dialogues between virtual patients and doctors and evaluated the diagnostic abilities of our system. We demonstrated that our system obtained impressive performance in both disease screening and differential diagnoses tasks. This research represents a step towards more seamlessly integrating AI into clinical settings, potentially enhancing the accuracy and accessibility of medical diagnostics.