MAGDA: Multi-agent guideline-driven diagnostic assistance

TL;DR

MAGDA introduces a multi-agent, guideline-driven approach using LLMs and vision-language models for zero-shot medical diagnosis, improving accuracy and adaptability in resource-limited settings.

Contribution

This work presents a novel multi-agent system that synthesizes diagnostic guidelines with vision-language models for zero-shot medical decision support.

Findings

Outperforms existing zero-shot methods on chest X-ray datasets.

Demonstrates generalizability to rare diseases.

Provides interpretable reasoning for diagnoses.

Abstract

In emergency departments, rural hospitals, or clinics in less developed regions, clinicians often lack fast image analysis by trained radiologists, which can have a detrimental effect on patients' healthcare. Large Language Models (LLMs) have the potential to alleviate some pressure from these clinicians by providing insights that can help them in their decision-making. While these LLMs achieve high test results on medical exams showcasing their great theoretical medical knowledge, they tend not to follow medical guidelines. In this work, we introduce a new approach for zero-shot guideline-driven decision support. We model a system of multiple LLM agents augmented with a contrastive vision-language model that collaborate to reach a patient diagnosis. After providing the agents with simple diagnostic guidelines, they will synthesize prompts and screen the image for findings following these guidelines. Finally, they provide understandable chain-of-thought reasoning for their diagnosis, which is then self-refined to consider inter-dependencies between diseases. As our method is zero-shot, it is adaptable to settings with rare diseases, where training data is limited, but expert-crafted disease descriptions are available. We evaluate our method on two chest X-ray datasets, CheXpert and ChestX-ray 14 Longtail, showcasing performance improvement over existing zero-shot methods and generalizability to rare diseases.