Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia

TL;DR

This study explores how large language models can generate diagnostic pathways for anemia, demonstrating their potential to support clinical decision-making and address limitations of traditional guidelines.

Contribution

The paper introduces a method for using advanced prompting techniques with LLMs to create diagnostic pathways, improving anemia diagnosis support.

Findings

GPT-4 outperforms other models in pathway accuracy

LLMs can generate realistic diagnostic sequences

Potential for LLMs to assist in clinical pathway discovery

Abstract

In practice, clinicians achieve a diagnosis by following a sequence of steps, such as laboratory exams, observations, or imaging. The pathways to reach diagnosis decisions are documented by guidelines authored by expert organizations, which guide clinicians to reach a correct diagnosis through these sequences of steps. While these guidelines are beneficial for following medical reasoning and consolidating medical knowledge, they have some drawbacks. They often fail to address patients with uncommon conditions due to their focus on the majority population, and are slow and costly to update, making them unsuitable for rapidly emerging diseases or new practices. Inspired by clinical guidelines, our study aimed to develop pathways similar to those that can be obtained in clinical guidelines. We tested three Large Language Models (LLMs) -Generative Pretrained Transformer 4 (GPT-4), Large Language Model Meta AI (LLaMA), and Mistral -on a synthetic yet realistic dataset to differentially diagnose anemia and its subtypes. By using advanced prompting techniques to enhance the decision-making process, we generated diagnostic pathways using these models. Experimental results indicate that LLMs hold huge potential in clinical pathway discovery from patient data, with GPT-4 exhibiting the best performance in all conducted experiments.