Agentifying Patient Dynamics within LLMs through Interacting with Clinical World Model

TL;DR

This paper introduces SepsisAgent, an LLM-based agent augmented with a Clinical World Model to improve sepsis treatment recommendations by simulating patient responses and learning from interactions.

Contribution

The paper presents a novel agent framework that combines LLMs with a learned world model for sequential clinical decision-making in sepsis management.

Findings

SepsisAgent outperforms traditional RL and LLM baselines in off-policy value.

Repeated interaction with the world model improves the agent's understanding of patient evolution.

SepsisAgent maintains performance even without simulator access.

Abstract

Sepsis management in the ICU requires sequential treatment decisions under rapidly evolving patient physiology. Although large language models (LLMs) encode broad clinical knowledge and can reason over guidelines, they are not inherently grounded in action-conditioned patient dynamics. We introduce SepsisAgent, a world model-augmented LLM agent for sepsis treatment recommendation. SepsisAgent uses a learned Clinical World Model to simulate patient responses under candidate fluid--vasopressor interventions, and follows a propose--simulate--refine workflow before committing to a prescription. We first show that world-model access alone yields inconsistent LLM decision performance, motivating agent-specific training. We then train SepsisAgent through a three-stage curriculum: patient-dynamics supervised fine-tuning, propose--simulate--refine behavior cloning, and world-model-based agentic reinforcement learning. On MIMIC-IV sepsis trajectories, SepsisAgent outperforms all traditional RL and LLM-based baselines in off-policy value while achieving the best safety profile under guideline adherence and unsafe-action metrics. Further analysis shows that repeated interaction with the Clinical World Model enables the agent to learn regularities in patient evolution, which remain useful even when simulator access is removed.