Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning

TL;DR

This paper introduces LA-CDM, a language agent that models iterative, hypothesis-driven clinical decision-making using reinforcement learning, improving diagnostic accuracy and efficiency in real-world datasets.

Contribution

It presents a novel hybrid training approach for language agents to support dynamic clinical diagnosis through iterative hypothesis testing.

Findings

LA-CDM improves diagnostic accuracy on MIMIC-CDM dataset.

Explicit training enhances clinical decision-making efficiency.

The approach models realistic, interactive diagnostic processes.

Abstract

Clinical decision-making is a dynamic, interactive, and cyclic process where doctors have to repeatedly decide on which clinical action to perform and consider newly uncovered information for diagnosis and treatment. Large Language Models (LLMs) have the potential to support clinicians in this process, however, most applications of LLMs in clinical decision support suffer from one of two limitations: Either they assume the unrealistic scenario of immediate availability of all patient information and do not model the interactive and iterative investigation process, or they restrict themselves to the limited "out-of-the-box" capabilities of large pre-trained models without performing task-specific training. In contrast to this, we propose to model clinical decision-making for diagnosis with a hypothesis-driven uncertainty-aware language agent, LA-CDM, that converges towards a diagnosis via repeatedly requesting and interpreting relevant tests. Using a hybrid training paradigm combining supervised and reinforcement learning, we train LA-CDM with three objectives targeting critical aspects of clinical decision-making: accurate hypothesis generation, hypothesis uncertainty estimation, and efficient decision-making. We evaluate our methodology on MIMIC-CDM, a real-world dataset covering four abdominal diseases containing various clinical tests and show the benefit of explicitly training clinical decision-making for increasing diagnostic performance and efficiency.