Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales

TL;DR

This paper introduces a prompt-based framework enabling large language models to perform clinical reasoning and diagnosis by generating rationales, demonstrating their reasoning ability with efficient training and evaluation methods.

Contribution

The work presents a novel reasoning-aware diagnosis framework that uses prompt-generated rationales for clinical reasoning, reducing annotation costs and improving interpretability.

Findings

LLMs can generate diagnostic rationales effectively.

The framework improves clinical diagnosis accuracy.

A new evaluation criteria for clinical rationales is proposed.

Abstract

Machine reasoning has made great progress in recent years owing to large language models (LLMs). In the clinical domain, however, most NLP-driven projects mainly focus on clinical classification or reading comprehension, and under-explore clinical reasoning for disease diagnosis due to the expensive rationale annotation with clinicians. In this work, we present a "reasoning-aware" diagnosis framework that rationalizes the diagnostic process via prompt-based learning in a time- and labor-efficient manner, and learns to reason over the prompt-generated rationales. Specifically, we address the clinical reasoning for disease diagnosis, where the LLM generates diagnostic rationales providing its insight on presented patient data and the reasoning path towards the diagnosis, namely Clinical Chain-of-Thought (Clinical CoT). We empirically demonstrate LLMs/LMs' ability of clinical reasoning via extensive experiments and analyses on both rationale generation and disease diagnosis in various settings. We further propose a novel set of criteria for evaluating machine-generated rationales' potential for real-world clinical settings, facilitating and benefiting future research in this area.