CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis

TL;DR

This paper introduces Chain-of-Diagnosis (CoD), a method that enhances interpretability and transparency in large language model-based medical diagnostics by mimicking a physician's reasoning process and providing confidence distributions.

Contribution

The paper presents CoD, a novel approach that transforms LLM diagnostics into an interpretable chain, enabling controllability and symptom inquiry, demonstrated through DiagnosisGPT for extensive disease diagnosis.

Findings

DiagnosisGPT outperforms other LLMs on diagnostic benchmarks.

CoD provides transparent reasoning pathways and confidence distributions.

The approach improves interpretability and controllability in medical diagnosis models.

Abstract

The field of medical diagnosis has undergone a significant transformation with the advent of large language models (LLMs), yet the challenges of interpretability within these models remain largely unaddressed. This study introduces Chain-of-Diagnosis (CoD) to enhance the interpretability of LLM-based medical diagnostics. CoD transforms the diagnostic process into a diagnostic chain that mirrors a physician's thought process, providing a transparent reasoning pathway. Additionally, CoD outputs the disease confidence distribution to ensure transparency in decision-making. This interpretability makes model diagnostics controllable and aids in identifying critical symptoms for inquiry through the entropy reduction of confidences. With CoD, we developed DiagnosisGPT, capable of diagnosing 9604 diseases. Experimental results demonstrate that DiagnosisGPT outperforms other LLMs on diagnostic benchmarks. Moreover, DiagnosisGPT provides interpretability while ensuring controllability in diagnostic rigor.