Multi-Stage Patient Role-Playing Framework for Realistic Clinical Interactions

TL;DR

This paper introduces Ch-PatientSim, a new Chinese patient simulation dataset, and a Multi-Stage Patient Role-Playing framework to enhance the realism and personalization of clinical interactions in LLMs.

Contribution

It presents the first Chinese patient simulation dataset and a novel multi-stage role-playing framework that improves the authenticity of model-generated clinical dialogues.

Findings

Models produce overly formal responses lacking personality

The MSPRP framework significantly improves simulation realism

Dataset augmentation enhances model performance

Abstract

The simulation of realistic clinical interactions plays a pivotal role in advancing clinical Large Language Models (LLMs) and supporting medical diagnostic education. Existing approaches and benchmarks rely on generic or LLM-generated dialogue data, which limits the authenticity and diversity of doctor-patient interactions. In this work, we propose the first Chinese patient simulation dataset (Ch-PatientSim), constructed from realistic clinical interaction scenarios to comprehensively evaluate the performance of models in emulating patient behavior. Patients are simulated based on a five-dimensional persona structure. To address issues of the persona class imbalance, a portion of the dataset is augmented using few-shot generation, followed by manual verification. We evaluate various state-of-the-art LLMs and find that most produce overly formal responses that lack individual personality. To address this limitation, we propose a training-free Multi-Stage Patient Role-Playing (MSPRP) framework, which decomposes interactions into three stages to ensure both personalization and realism in model responses. Experimental results demonstrate that our approach significantly improves model performance across multiple dimensions of patient simulation.